Upgrading Homebrew and avoiding the "failed to verify attestation" error(til.simonwillison.net)
til.simonwillison.net
Upgrading Homebrew and avoiding the "failed to verify attestation" error
https://til.simonwillison.net/homebrew/no-verify-attestations
45 comments
My issues with homebrew are:
1. I hate the concept of dependency management. I want every package to ship with all dependencies inside. Just download tarball, extract and that's about it.
2. homebrew often wants to install things I already have, like python.
3. No easy way to install old packages.
I don't understand why things are made harder than they should be.
1. I hate the concept of dependency management. I want every package to ship with all dependencies inside. Just download tarball, extract and that's about it.
2. homebrew often wants to install things I already have, like python.
3. No easy way to install old packages.
I don't understand why things are made harder than they should be.
The downside of the "bundle everything" approach (which is also used by Docker and it's ilk), is that whenever one of those dependencies needs to be fixed or upgraded (for reliability or security reasons), you have to find every instance of it on the entire system, which soon becomes an extremely difficult task.
Shared libraries don't have this problem. Yes, they're separate packages, but having dependencies that can be upgraded separately simplifies upgrading that dependency.
Shared libraries don't have this problem. Yes, they're separate packages, but having dependencies that can be upgraded separately simplifies upgrading that dependency.
This is assuming that just upgrading the shared library will work for everything. Too often, some things are broken by the upgrade, and since you weren't explicitly trying to update the thing that broke, you might not notice until a later date, at which point you may struggle to remember what was updated that it relies on.
I don't disagree that Hyrum's Law[1] is definitely a thing, but in practice with libraries that attempt SemVer or similar compatibility guarantees and understand that they'll be used in a shared library environment, breakage is not that common.
It also doesn't work for some ecosystems (like Go) where the practice is to prefer static linking.
1: https://www.hyrumslaw.com
It also doesn't work for some ecosystems (like Go) where the practice is to prefer static linking.
1: https://www.hyrumslaw.com
> The downside of the "bundle everything" approach (which is also used by Docker and it's ilk), is that whenever one of those dependencies needs to be fixed or upgraded (for reliability or security reasons), you have to find every instance of it on the entire system, which soon becomes an extremely difficult task.
How it becomes difficult task? Just download things and replace them, when I ask to update. I have fast internet and big SSD, that's fine for me. 90% of software I'm using on my Mac are installed via alternative ways and they already bundle all the dependencies, so I already living with it.
How it becomes difficult task? Just download things and replace them, when I ask to update. I have fast internet and big SSD, that's fine for me. 90% of software I'm using on my Mac are installed via alternative ways and they already bundle all the dependencies, so I already living with it.
There are also ways to abstract the files on disk such that it appears every module has its own copy of “foo.so” but they’re all the same bytes on disk. Using content hashes for example. I believe this is how pnpm works.
I don’t buy the shared libraries solve problems argument either. Lots of software are pinned to a specific version anyway so just because some security update has come out for a shared lib doesn’t mean it will work with all your other software.
I don’t buy the shared libraries solve problems argument either. Lots of software are pinned to a specific version anyway so just because some security update has come out for a shared lib doesn’t mean it will work with all your other software.
Dependencies are totally out of control, in much of OSS, not just Homebrew. I probably have 5-6 leaf node packages that I actually installed with brew install, but brew list shows close to 100 little dependencies. Same with apt on my Ubuntu/Debian systems. I physically installed a handful of applications but my system has hundreds of packages installed. At least it’s not DLL Hell: the dependency management tools are great and uninstallation is clean, but wow what an explosion of quantity of dependencies!
I don’t use node but I understand it’s a mess there too.
I don’t use node but I understand it’s a mess there too.
But it's nice to reuse code
Isn't it nice to reuse code so you dont have to write it twice?
It doesn't mean we can't run it twice.
It doesn't mean we can't run it twice.
You might be able to a nice graph with this command: brew deps --installed --graph
There’s command to see your leaf dependencies:
https://docs.brew.sh/Manpage#leaves---installed-on-request--...
https://docs.brew.sh/Manpage#leaves---installed-on-request--...
Yep I use that a lot. I like to make sure I only have installed what I need to survive, and uninstall everything else.
>2. homebrew often wants to install things I already have, like python.
I think its important to understand why this is the case. The python you think you have already, out of the box in MacOS, is the system python. Its not the python you should be using - its the one that python-based tools that your system depends on, is using.
Brew installs other versions of python - and gives you access to tools that allow you to maintain completely independent, different versions of python - for a very good reason.
You simply should not be using the system python for tools that are outside the purview of the system tools - doing so can lead to broken essential system tools.
So, don't be so quick to resist this aspect of package management. Its also true of Linux, by the way - developers should be using their own python installations, and not just glomming libraries into the system-provided python tree .. to do so, is to live very dangerously as a systems operator and as well as a developer.
I think its important to understand why this is the case. The python you think you have already, out of the box in MacOS, is the system python. Its not the python you should be using - its the one that python-based tools that your system depends on, is using.
Brew installs other versions of python - and gives you access to tools that allow you to maintain completely independent, different versions of python - for a very good reason.
You simply should not be using the system python for tools that are outside the purview of the system tools - doing so can lead to broken essential system tools.
So, don't be so quick to resist this aspect of package management. Its also true of Linux, by the way - developers should be using their own python installations, and not just glomming libraries into the system-provided python tree .. to do so, is to live very dangerously as a systems operator and as well as a developer.
> I want every package to ship with all dependencies inside.
Stopping where? python? c libraries? glibc? the kernel?
"all the dependencies" isn't what you think it is
> 2. homebrew often wants to install things I already have, like python.
oh yeah "python" like it's just A Thing You have. nothing has versions and of course every version can execute every code that's ever been written, past and future.
> I don't understand why things are made harder than they should be.
You're just willfully ignoring or not understanding the complexities.
Stopping where? python? c libraries? glibc? the kernel?
"all the dependencies" isn't what you think it is
> 2. homebrew often wants to install things I already have, like python.
oh yeah "python" like it's just A Thing You have. nothing has versions and of course every version can execute every code that's ever been written, past and future.
> I don't understand why things are made harder than they should be.
You're just willfully ignoring or not understanding the complexities.
> Stopping where?
Where OS provides guarantees. If OS provides guarantee that libc will be there, do not ship libc. If OS provides guarantees that python will be there, do not ship python. If you do ship python, hide it very well, so I'd never even know about it, unless I go out of my way. And it'll never be shared by anything.
Those questions are easy and solved by every commercial software. They need to make those choices and they do make it.
Where OS provides guarantees. If OS provides guarantee that libc will be there, do not ship libc. If OS provides guarantees that python will be there, do not ship python. If you do ship python, hide it very well, so I'd never even know about it, unless I go out of my way. And it'll never be shared by anything.
Those questions are easy and solved by every commercial software. They need to make those choices and they do make it.
I'm sure you're familiar with versions. What happens when your software depends on a libc that has a function that was removed on the newer version, or added since the previous one? Now older or newer versions of libc don't work with your software, even though they're "there".
Right now I'm downloading "sonoma_bottle" build through the homebrew. So they already build versions for every supported macos version. I'm pretty sure that macos libc will not remove functions in the minor upgrade, and having 2-3 builds for currently supported OSes is not a big deal.
> 2. homebrew often wants to install things I already have, like python.
Well macOS does ship with Python but it's a true pain in the rear to deal with version conflicts of global packages.
Well macOS does ship with Python but it's a true pain in the rear to deal with version conflicts of global packages.
My biggest gripe: You can only have one homebrew user per machine, so on a multi-user, multi-admin Mac, only one of those is allowed to brew install.
Yeah, their shitty chown system folders process is a bit shit.
I have had a small amount of success installing in home, when I had a locked down machine for work but it really wants to fight it.
I have had a small amount of success installing in home, when I had a locked down machine for work but it really wants to fight it.
I just switched over to devbox after reading this thread, I like it a lot so far, thanks.
MacPorts is a great alternative, also way older but never has been as successful as Homebrew.
Second recommendation for MacPorts
It predates Homebrew by a bit and is under Apple's http://www.macosforge.org umbrella of OSS projects, so as close to 1st party support as you can get.
It predates Homebrew by a bit and is under Apple's http://www.macosforge.org umbrella of OSS projects, so as close to 1st party support as you can get.
MacPorts rocks. No ports breaking when upgrading MacOS as can (or could,not sure if that's still an issue) with Brew.
[deleted]
Third one for macports. It's more sane, at least for multi users support (I create an user for each contract work)
It seems like Apple also closely work with Homebrew - https://news.ycombinator.com/item?id=41708046#41711168
MacPorts is great!
Another excellent alternative is pkgsrc: https://pkgsrc.smartos.org/install-on-macos/
I've used pkgsrc on SmartOS/illumos and NetBSD for many years, but this is my first time using it on macOS -- about 18 months now, and all experiences positive!
Another excellent alternative is pkgsrc: https://pkgsrc.smartos.org/install-on-macos/
I've used pkgsrc on SmartOS/illumos and NetBSD for many years, but this is my first time using it on macOS -- about 18 months now, and all experiences positive!
Homebrew is really the worst of all the options on macOS, I wonder how it became so popular in the first place.
What I have learned from Nix is that polluting the global (user or system) environment is going to spell disaster at some point. Its obviously fixable in an obvious way, but why subject yourself to this nonsense in the first place. The only stuff I have in my global env is stuff I could actually use from almost anywhere, even non-code directories.
Nix isn't the only tool that solves this, there is acme and even per-compiler tools like nvm or cargo.
Nix isn't the only tool that solves this, there is acme and even per-compiler tools like nvm or cargo.
In case it shouldn't work out, I never had any problems with MacPorts.
I will have a look at devbox. I always found Nix on macOS too much hassle, maybe the situation has gotten better?
I will have a look at devbox. I always found Nix on macOS too much hassle, maybe the situation has gotten better?
All I can say when I see Homebrew: "I replaced Homebrew with Devbox" [1]
[1] https://mootoday.com/blog/i-replaced-homebrew-with-devbox
[1] https://mootoday.com/blog/i-replaced-homebrew-with-devbox
Some folks are running nix on OSX along with nix-darwin.
https://github.com/LnL7/nix-darwin
https://github.com/LnL7/nix-darwin
nix-darwin is the cherry on the cake, plain nixpkgs is already miles better than homebrew.
> I frequently have rate limiting problems with my GitHub account
What are people doing to get their account rate limited?
What are people doing to get their account rate limited?
I still haven't figured out quite what's going on there, but I wouldn't be surprised to learn that it was due to all of the scrapers I'm running: https://simonwillison.net/2020/Oct/9/git-scraping/
If you are in a large company that VPNs all your traffic to one point, you can get rate limited if you visit GitHub without logging in.
If you are getting the GitHub API, the rate limit is 60 request per hour, per IP. That is easy to hit. If you send an API token, it’s 5000 requests per hour.
https://docs.github.com/en/rest/using-the-rest-api/rate-limi...
If you are getting the GitHub API, the rate limit is 60 request per hour, per IP. That is easy to hit. If you send an API token, it’s 5000 requests per hour.
https://docs.github.com/en/rest/using-the-rest-api/rate-limi...
CI/CD pipelines at large scale can pull directly from src repos sometimes causing stuff like this, just off the top of my head. I think the default is 5000 per hour which is not very much for automation to chew through. I ran into something similar with bitbucket once that I had to engineer around because it was very annoying.
[deleted]
1. Delete Homebrew
2. Install Macports
2. Install Macports
I ran in to the same problem last weekend and got this suggested fix from gpt-o1.
I did try to Google it first, but wasn't able to find anything on it.
I did try to Google it first, but wasn't able to find anything on it.
[deleted]
Brew has always been dogshit. Nobody runs macos in production. Please stop using this.
As someone who hates tinkering with this kinda stuff, I’m surprised how well it works so far