Go mod’s lesser known features(verdverm.com)
verdverm.com
Go mod’s lesser known features
https://verdverm.com/go-mods/
28 comments
It could be worse, it could be Java. I might be wrong here, but as I understand it: Java’s use of the terms module and package which these days vary depending on context. Once upon a time, you put modules of code in a namespaced package, but then these packages were distributed and run as modules, and suddenly both package and module ceased to have clear meanings - context is required.
Then again, maybe we should make up new terms instead of re-using existing ones. Like Ruby “Gems” or Homebrew’s “Casks”. It’s not intrinsically descriptive, but it is unique and unambiguous. I suppose again it could be worse, we could be talking about “libraries” and “frameworks” while describing “modules” and “packages”.
Then again, maybe we should make up new terms instead of re-using existing ones. Like Ruby “Gems” or Homebrew’s “Casks”. It’s not intrinsically descriptive, but it is unique and unambiguous. I suppose again it could be worse, we could be talking about “libraries” and “frameworks” while describing “modules” and “packages”.
Those packages were distributed and run as JAR, WAR or EAR files.
I never remember having seen them referenced as modules.
I never remember having seen them referenced as modules.
OSGi modules? It’s true there are other names, but if you Google “Java 9 packages modules” you’ll get some very weird results. For example: https://softwareengineering.stackexchange.com/questions/3112...
OSGi modules are related to Eclipse and how they modularized their components.
Yes modules in Java don't mean the same thing as in Modula-2, Ada has packages, child packages and nested packages, Object Pascal/UCSD Pascal has units, while VMS Pascal has modules, and so on.
There is plenty of literature to educate onself about the subject instead of Internet random searches.
Yes modules in Java don't mean the same thing as in Modula-2, Ada has packages, child packages and nested packages, Object Pascal/UCSD Pascal has units, while VMS Pascal has modules, and so on.
There is plenty of literature to educate onself about the subject instead of Internet random searches.
I dunno, I feel like 'module' communicates something that is only code, while 'package' - but this is my JS experience speaking - implies it may also contain non-code files like docs, tests, examples, configuration files, html, etc that aren't directly used by the application importing them.
Modules = Lego bricks used to build bigger Lego artifacts
Package = Lego bricks (probably share a theme and are meant to build something concrete together), shipped with instructions and media and whatnot, in a, well, whole package.
Package = Lego bricks (probably share a theme and are meant to build something concrete together), shipped with instructions and media and whatnot, in a, well, whole package.
go modules is one of the hardest things about learning go as a new learner. most of the documentation talks of gopath. the go mod error messages are extremely cryptic. why are go mod vendor and go mod tidy different commands?
the lack of “one true way” is very strange. some projects module. some projects check in dependent source code. some projects have a version number in the package name. do i download with go get? or go install? or go get -u?
i can’t think of another build system that has so much variation just in declaring and importing dependencies
the lack of “one true way” is very strange. some projects module. some projects check in dependent source code. some projects have a version number in the package name. do i download with go get? or go install? or go get -u?
i can’t think of another build system that has so much variation just in declaring and importing dependencies
I can shed some light on some of your questions for you.
> most of the documentation talks of gopath
Historically go used a GOPATH env var. Go code had to live under there under `$GOPATH/src/<module-path>`. When doing an import, go would look at the import path, and find it in GOPATH. This wasn't flexible enough but still exists under the hood as it's where go saves (caches?) packages globally. If you have a go.mod/go.sum file in your project, in general, you won't have to worry about GOPATH.
Worth noting that go, while widely used, was created to solve Google's problems (I guess at least initially?), where (from my understanding) they have a huge monorepo. At that point, having everything live under such a GOPATH somewhat makes sense.
> why are go mod vendor and go mod tidy different commands?
Go mod vendor puts all the dependencies in a ./vendor directory. Often used to commit your packages directly into git. There might be other advantages, not sure, but personally I don't use it. Go is able to pick up the packages in your go.mod directly from $GOPATH.
Go mod tidy ensures your go.mod matches your used package. E.g. if you remove usage of a package in your code, go mod tidy is able to pick that up. In addition, it also ensures your go.sum matches go.mod.
> the lack of “one true way” is very strange. some projects module. some projects check in dependent source code
This is a historic issue. Prior to 1.12 (I think), go modules didn't exist, and there were different community projects that attempted to solve go package management.
> do i download with go get? or go install? or go get -u?
Again, historic issues, iirc go get -u forces an update of a package, meaning this updates your go.mod file. Go get without -u does not force update the package. It update your go.mod file if you haven't included the module previously. Go install is used for go main packages. It fetches the code, builds it (so it must be a main package), and puts the final binary in $GOPATH/bin. Assuming you have that in your patch, you can use it straight away.
> most of the documentation talks of gopath
Historically go used a GOPATH env var. Go code had to live under there under `$GOPATH/src/<module-path>`. When doing an import, go would look at the import path, and find it in GOPATH. This wasn't flexible enough but still exists under the hood as it's where go saves (caches?) packages globally. If you have a go.mod/go.sum file in your project, in general, you won't have to worry about GOPATH.
Worth noting that go, while widely used, was created to solve Google's problems (I guess at least initially?), where (from my understanding) they have a huge monorepo. At that point, having everything live under such a GOPATH somewhat makes sense.
> why are go mod vendor and go mod tidy different commands?
Go mod vendor puts all the dependencies in a ./vendor directory. Often used to commit your packages directly into git. There might be other advantages, not sure, but personally I don't use it. Go is able to pick up the packages in your go.mod directly from $GOPATH.
Go mod tidy ensures your go.mod matches your used package. E.g. if you remove usage of a package in your code, go mod tidy is able to pick that up. In addition, it also ensures your go.sum matches go.mod.
> the lack of “one true way” is very strange. some projects module. some projects check in dependent source code
This is a historic issue. Prior to 1.12 (I think), go modules didn't exist, and there were different community projects that attempted to solve go package management.
> do i download with go get? or go install? or go get -u?
Again, historic issues, iirc go get -u forces an update of a package, meaning this updates your go.mod file. Go get without -u does not force update the package. It update your go.mod file if you haven't included the module previously. Go install is used for go main packages. It fetches the code, builds it (so it must be a main package), and puts the final binary in $GOPATH/bin. Assuming you have that in your patch, you can use it straight away.
GOPATH is arguably more an artifact of Plan9 than google3 monorepo, with Plan9 kinda ending up in monorepo-style access with public replica servers linked at /n/
Ugh, reading this reminded me why I felt good moving away from Go after the honeymoon period was over.
Actually writing the code was fun at first, but everything else around the actual act of composing code was thoroughly awful UX.
Actually writing the code was fun at first, but everything else around the actual act of composing code was thoroughly awful UX.
It looks complicated because OP is describing the situations both before and after the introduction of go mod.
Just use go get to get initially, and go mod to organize thereafter. It is very simple and occupies <1% of my cognitive effort writing go code.
Just use go get to get initially, and go mod to organize thereafter. It is very simple and occupies <1% of my cognitive effort writing go code.
I'm the opposite: I'm writing Go for a living and I hate everything about the language itself, but I found the de-facto toolchain ok-ish and probably the least obtrusive thing about the whole thing.
Imho, many languages would massively benefit with the kind of tooling Go has out of the box. I heard Rust's tooling has some great stuff too and it's in my immediate plan to learn some Rust.
Imho, many languages would massively benefit with the kind of tooling Go has out of the box. I heard Rust's tooling has some great stuff too and it's in my immediate plan to learn some Rust.
I've had a similar experience, though my pain points were mostly around using privately hosted modules internally.
I forget the solution, but involved messing about with the `GOPRIVATE` env variable, adding some `.insteadOf` entries to git config, plus similar fun trying to get the same happening in a CI environment.
I forget the solution, but involved messing about with the `GOPRIVATE` env variable, adding some `.insteadOf` entries to git config, plus similar fun trying to get the same happening in a CI environment.
I'd love to understand what you think the issue is. I feel like go dependencies have been extremely easy since modules were introduced and I've rarely had issues.
Can you elaborate? It seems to me like the minimum set of features I want from package management
I have a go thing I wrote that I hadn't touched for years but had the misfortune to need to modify it recently, and the way to migrate from $GOPATH/src/ to go.mod is not really clear (to me at least). To build this thing, the only go code I have, I used git submodules to put all the dependencies in ./src, so I had ./src/github.com and a load of 3rd party modules in there, then my code was in ./src/Mything. To build it, I ran "env GOPATH=$PWD go build Mything/MyPackage" and it did the right thing, with the final binary appearing in the current directory.
Now this gives an error "go.mod file not found in current directory", but if I add that file, go build says "unexpected go.mod file found in current directory".
Luckily, for now, I can use GO111MODULE=off to keep pretending everything is fine, but my eyes start to glaze over when I read the migration guide at https://go.dev/blog/migrating-to-go-modules - it's one of those moments where I start to wish I'd used a different language :(
Now this gives an error "go.mod file not found in current directory", but if I add that file, go build says "unexpected go.mod file found in current directory".
Luckily, for now, I can use GO111MODULE=off to keep pretending everything is fine, but my eyes start to glaze over when I read the migration guide at https://go.dev/blog/migrating-to-go-modules - it's one of those moments where I start to wish I'd used a different language :(
I’m pretty sure the migration is just
1. Go mod init in your code’s root dir 2. Done..? Go get ./…?
1. Go mod init in your code’s root dir 2. Done..? Go get ./…?
Nah, I wish it were it that easy. There is definitely more yak shaving involved.
"go mod init" returns the error "go: cannot determine module path for source directory".
"go mod init src/mything/mypackage" creates a go.mod file, but then "env GOPATH=$PWD go build mything/mypackage" says "$GOPATH/go.mod exists but should not" (as opposed to when it isn't there, then go complains "go.mod file not found in current directory")
Note that "env GOPATH=$PWD GO111MODULE=off go build mything/mypackage" does build all the source code.
Most likely I've set the whole project and directory structure up wrong, and it only works by brute force.
"go mod init" returns the error "go: cannot determine module path for source directory".
"go mod init src/mything/mypackage" creates a go.mod file, but then "env GOPATH=$PWD go build mything/mypackage" says "$GOPATH/go.mod exists but should not" (as opposed to when it isn't there, then go complains "go.mod file not found in current directory")
Note that "env GOPATH=$PWD GO111MODULE=off go build mything/mypackage" does build all the source code.
Most likely I've set the whole project and directory structure up wrong, and it only works by brute force.
it should have been `go mod init mything/mypackage` (without the `src/`) in the `GOPATH/src/mything/mypackage` directory,
and `GOPATH=$PWD` is almost certainly wrong. The `go` command in module mode assumes independence from GOPATH, so to build something, you have to be "in" the module (under a directory with a `go.mod` file).
Thanks, I've been checking on project layouts, and I have mine definitely wrong / weird. Which is probably why I've made life difficult for myself.
go mod tidy and go mod vendor are different commands because they do very different things.
> Requires a domain name to be the first part of the module identifier
This is only true if you want the module to be publicly 'go get'-able. Private modules can be named whatever you want.
(Some tools use whether or not the first import path segment contains a '.' as a heuristic for "is this package stdlib", and those won't work correctly on a module that doesn't use a dot. There's a proposal, not yet accepted, to document this as a naming requirement for modules: https://github.com/golang/go/issues/32819 This is of course a looser requirement than "must be a domain name".)
This is only true if you want the module to be publicly 'go get'-able. Private modules can be named whatever you want.
(Some tools use whether or not the first import path segment contains a '.' as a heuristic for "is this package stdlib", and those won't work correctly on a module that doesn't use a dot. There's a proposal, not yet accepted, to document this as a naming requirement for modules: https://github.com/golang/go/issues/32819 This is of course a looser requirement than "must be a domain name".)
One feature not listed is that the proxy caches 404 (which is probably reasonable), but the consequence is if you try to build with the new version before pushing the dependency, you won't be able to build without GONOPROXY until the cache expires. (Update dep code, test with go.mod replace =>, tag dep, (forget push), remove replace =>, boops.)
I literally ran into this 3 hours ago and couldn't figure out why. That explains it! Thanks for the insight!
workspace of Go1.18 may solve your issues
https://sebastian-holstein.de/post/2021-11-08-go-1.18-featur...
I found this article in the discussion here (https://news.ycombinator.com/item?id=30869261). I thought it was worth posting because that discussion seemed to contain a lot of confusion & misinformation about go modules, so there seems to be some need for clarification!
If you the one or know them there is a spelling error:
but also for security resaons. << reasons not resaons
but also for security resaons. << reasons not resaons
I wish Go had got these names the right way round. Every other programming language and packaging (NB. not "modeling"!) ecosystem uses package to mean distribution, and module to mean compilation unit.