Caddy's build are reproducible as per https://github.com/golang/go/wiki/Modules. The downloader offers you to extend Caddy with plugins, and this will change the hash of the binary compared to a straight build from the source code. Keep in mind that Telemetry is enabled by default in the source code, unlike @ the downloader, this changes the hash too.