Inspecting the Source of Go Modules(words.filippo.io)
words.filippo.io
Inspecting the Source of Go Modules
https://words.filippo.io/go-source/
26 comments
See also https://github.com/golang/go/issues/66653 (that I don’t think is linked from the article)
> but it is better understood as exploiting the natural lack of verification in the GitHub web interface, which doesn’t show the authentic (and in this case malicious) source of a module version, as used by actual Go tooling.
That's a load of crap. It can never become a github issue that the Go ecosystem has chosen to make it look like the packages you pull come from github, but are actually diverted to be served by Google from some strange pull-through cache.
Calling this a "lack of verification in the GitHub web interfaces" completely inverts the abstraction layers and asks GitHub to implement specific features for your incorrect usecase.
Then there's the misnomer of a hash database being anywhere close to analogous to PGP signed sources. This is amateur level stuff.
That's a load of crap. It can never become a github issue that the Go ecosystem has chosen to make it look like the packages you pull come from github, but are actually diverted to be served by Google from some strange pull-through cache.
Calling this a "lack of verification in the GitHub web interfaces" completely inverts the abstraction layers and asks GitHub to implement specific features for your incorrect usecase.
Then there's the misnomer of a hash database being anywhere close to analogous to PGP signed sources. This is amateur level stuff.
FTA:
> “For example, there is no guarantee that the code displayed at https://github.com/example/mod/blob/v1.2.3/exp.go is the actual contents of exp.go from v1.2.3 of module github.com/example/mod: GitHub allows force-pushing git tags and even built its recommended GitHub Actions workflows on top of mutable tags.”
In a sense this is a git issue: the fact that git is mutable instead of append-only. Git wasn’t designed to serve as part of a code integrity system - you’d have to add that at some other level, such as Github.
Or in the case of this example, recognize that tags can’t be secured and implement everything in terms of commit hashes, etc.
> “For example, there is no guarantee that the code displayed at https://github.com/example/mod/blob/v1.2.3/exp.go is the actual contents of exp.go from v1.2.3 of module github.com/example/mod: GitHub allows force-pushing git tags and even built its recommended GitHub Actions workflows on top of mutable tags.”
In a sense this is a git issue: the fact that git is mutable instead of append-only. Git wasn’t designed to serve as part of a code integrity system - you’d have to add that at some other level, such as Github.
Or in the case of this example, recognize that tags can’t be secured and implement everything in terms of commit hashes, etc.
> the fact that git is mutable instead of append-only
Remind me again how I change a commit in the middle of a branch without affecting the commits that come later in the same branch?
Remind me again how I change a commit in the middle of a branch without affecting the commits that come later in the same branch?
git rebase can do that easily, I do it all the time when prepping feature branches for submission to the main branch
> git rebase can do that easily
What exact parameters you use for not changing the latter commits? Must be a very new feature, git's whole idea used to be that commits are on top of the previous ones, rewriting one needs rewrite of all the latter ones.
What exact parameters you use for not changing the latter commits? Must be a very new feature, git's whole idea used to be that commits are on top of the previous ones, rewriting one needs rewrite of all the latter ones.
Instead of making a sarcastic response, just state plainly what you mean. This prevents you having to get into dumb arguments that don't mean anything to people who don't already know what you're trying to say.
literally just 'git rebase origin/main' to insert new commits to main into my branch history prior to the commits I made. No rewrites of later commits required if there are no conflicts. You seem to have an extreme misunderstanding of how git works.
Talking about extreme misunderstandings, rebase always rewrites commits, even when there are conflicts. Look at the commit sha's next time you're doing that.
> implement everything in terms of commit hashes
And remember, git uses a vulnerable algorithm :)
And remember, git uses a vulnerable algorithm :)
You are right. GitHub has nothing to do with this problem — it's entirely on Go.
> Go has indisputably the best package integrity story of any programming language ecosystem
Nope, still catching up to Java and .NET.
Nope, still catching up to Java and .NET.
I would love to learn more. What's the package integrity story of Java and .NET?
All I can find is documentation about artifacts on e.g. Maven Central being signed with any PGP key, which can freely change across package versions. If that's correct, it's no more than a convoluted checksum (without anything resembling the Checksum Database and its transparency log). If that's not correct, I am very curious what the workflow is when a package author loses a key.
Or, more concretely: what's stopping Maven Central from serving a fake version of someone else's package to a targeted victim?
All I can find is documentation about artifacts on e.g. Maven Central being signed with any PGP key, which can freely change across package versions. If that's correct, it's no more than a convoluted checksum (without anything resembling the Checksum Database and its transparency log). If that's not correct, I am very curious what the workflow is when a package author loses a key.
Or, more concretely: what's stopping Maven Central from serving a fake version of someone else's package to a targeted victim?
I've always found it very weird that GitHub's tolerated being completely subverted to become a package manager for ecosystems that can't be bothered to manage it themselves.
NPM doesn't just proxy to GitHub (even though both are owned by Microsoft).
To see maintainers criticise GitHub for not being a perfect package manager is lunacy.
See also https://github.com/CocoaPods/CocoaPods/issues/4989
NPM doesn't just proxy to GitHub (even though both are owned by Microsoft).
To see maintainers criticise GitHub for not being a perfect package manager is lunacy.
See also https://github.com/CocoaPods/CocoaPods/issues/4989
There is no criticism of GitHub in the post, aside from throwing a bit of shade at them using mutable git tags for Actions instead of actually building a package manager.
The lack of verification of ecosystem-specific authenticity is natural, as the post says, in reading source directly from any code host.
NPM has the same problem if you click through to the source repository and expect what you read to match the package. It’s been used to hide attacks in that ecosystem in the same way, and the NPM web UI recently added a code browser similar to the one in this post.
If anything, the extra upload step of NPM (and similar centralized registries) makes things worse by encouraging and normalizing publishing different source from what is in the VCS.
(Also, Go doesn't use GitHub as a package manager. It's just one of the many supported code hosts. In fact, anything that can serve a VCS repo or a zip file is supported.)
The lack of verification of ecosystem-specific authenticity is natural, as the post says, in reading source directly from any code host.
NPM has the same problem if you click through to the source repository and expect what you read to match the package. It’s been used to hide attacks in that ecosystem in the same way, and the NPM web UI recently added a code browser similar to the one in this post.
If anything, the extra upload step of NPM (and similar centralized registries) makes things worse by encouraging and normalizing publishing different source from what is in the VCS.
(Also, Go doesn't use GitHub as a package manager. It's just one of the many supported code hosts. In fact, anything that can serve a VCS repo or a zip file is supported.)
That’s fair, but GitHub themselves do it with GitHub Actions. Versioning of all official actions is done with git tags, which has always made me uncomfortable.
> GitHub themselves do it with GitHub Actions
That makes sense, so does doing releases by using tags, why would that make you uncomfortable?
What doesn't make sense, is creating a completely new language/framework/package manager and decide to place the package registry burden on someone else.
That makes sense, so does doing releases by using tags, why would that make you uncomfortable?
What doesn't make sense, is creating a completely new language/framework/package manager and decide to place the package registry burden on someone else.
> That makes sense, so does doing releases by using tags, why would that make you uncomfortable?
Tags are not immutable.
Tags are not immutable.
Are you confusing tags with branches? Git tags for all intents and purposes are immutable. If you have a tag in your local repository that was pulled down from another remote, and the remote changes the tag and you update your local repository, it'll reject the incoming tag.
It's true they're both refs inside git, but git literally treats them as "shouldn't move", unlike branches. They're not immutable in the technical sense, so I guess you're technically right. But they're not used the same way as branches, and the tooling won't like that either.
It's true they're both refs inside git, but git literally treats them as "shouldn't move", unlike branches. They're not immutable in the technical sense, so I guess you're technically right. But they're not used the same way as branches, and the tooling won't like that either.
That's irrelevant in the context of GitHub Actions: CI runners clone the repo fresh every run. If someone gets write access to a widely used action, they can force push all tags to point to a malicious commit.
This is even in GitHub's docs: https://docs.github.com/en/actions/reference/security/secure...
> Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
This is even in GitHub's docs: https://docs.github.com/en/actions/reference/security/secure...
> Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
If you're in (for example) a CI context and do a git checkout @tag, there's no guarantee that you'll get the same content as the last time you fetched that tag.
Tags are not immutable.
Tags are not immutable.
Go does not use Github as a package manager.
They love to be essential and central even if it costs them. Same reason why they won't ratelimit 10000000000000x consecutive requests from the same source on the same resource on npm.
No, services that lean too heavily on Github routinely receive messages from the admins asking them to do what they can to dial back usage of server resources. It happened to Homebrew, it happened to Rust, and it happened to various other projects who thought they were being good citizens by recommending shallow clones (turns out, shallow clones are somehow harder on the server than full clones).