Checksum in sum.golang.org is different from download without proxy/checksum database (GOPRIVATE=*) about berglas HOT 7 CLOSED

Alright, I reworked your example a bit to eliminate as many variables as possible and was still able to reproduce it (workflow).

I downloaded the cached binary from https://proxy.golang.org/github.com/%21google%21cloud%21platform/berglas/@v/v0.6.2.zip, unzipped it, and compared it to the v0.6.2 tag. It looks like none of the examples/ are included:

HEAD detached at v0.6.2
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	deleted:    examples/appengine/go/.gcloudignore
	deleted:    examples/appengine/go/.gitignore
	deleted:    examples/appengine/go/README.md
	deleted:    examples/appengine/go/app.yaml
	deleted:    examples/appengine/go/go.mod
	deleted:    examples/appengine/go/go.sum
	deleted:    examples/appengine/go/main.go
	deleted:    examples/appengineflex/go/.gitignore
	deleted:    examples/appengineflex/go/Dockerfile
	deleted:    examples/appengineflex/go/README.md
	deleted:    examples/appengineflex/go/app.yaml
	deleted:    examples/appengineflex/go/go.mod
	deleted:    examples/appengineflex/go/go.sum
	deleted:    examples/appengineflex/go/main.go
	deleted:    examples/cloudfunctions/go/README.md
	deleted:    examples/cloudfunctions/go/fn.go
	deleted:    examples/cloudfunctions/go/go.mod
	deleted:    examples/cloudfunctions/go/go.sum
	deleted:    examples/cloudrun/go/Dockerfile
	deleted:    examples/cloudrun/go/README.md
	deleted:    examples/cloudrun/go/go.mod
	deleted:    examples/cloudrun/go/go.sum
	deleted:    examples/cloudrun/go/main.go
	deleted:    examples/kubernetes/README.md
	deleted:    examples/kubernetes/deploy/sample.yaml
	deleted:    examples/kubernetes/deploy/webhook.yaml
	deleted:    examples/kubernetes/go.mod
	deleted:    examples/kubernetes/go.sum
	deleted:    examples/kubernetes/main.go

This makes me think that the proxy doesn't include that directory, either because it declares its own go.sum/mod files, or because it is named examples (which seems weird).

Nonetheless, I don't see anything malicious here. If you believe the Go module proxy is modifying packages in ways that it shouldn't, I would recommend using the contact information at the bottom of https://proxy.golang.org/.

from berglas.

Would the rename from examples/ to demo/ will fix this problem?

It's possible, but I "examples" is a pretty common name. I really think a bug should be opened against the module registry for this.

from berglas.

I have no idea where to even begin to debug something like that.

from berglas.

I tried running the script in your gist on 3 different networks (home, cellular, coffeeshop) to try and reproduce.

First, I tried with no other dependencies besides berglas (removed "github.com/sirupsen/logrus"). On all three networks, I go no diff.

Next, I tried adding "github.com/sirupsen/logrus" back in, but got the same result.

My suspicion is that your proxy, network, ISP, or something higher up the chain is modifying the response.

from berglas.

Logrus has nothing to do with the issue. It presents a completely other dependency that does not seem to have different checksum, whether the GOPRIVATE=* is set or not. It also takes part in verification if my network setup has some proxy or doesn't. Because, the checksum is exact for both scenarios, I think I don't have any unknown proxy of my ISP.

Also, because we start blaming someone's other network, which likely has nothing to do with my problem, I have prepared you two GitHub workflows, one with GOPRIVATE=* and other without. I find GitHub-managed workflow runners as a neutral ground and good place for more investigation. Here are results, which confirms my thesis - I am not only one, who can be haunted by this problem:

Without GOPRIVATE

• Workflow definition: https://github.com/piotrkubisa/berglas-checksum/blob/master/.github/workflows/no-goprivate.yml#L6-L9
• Test run: https://github.com/piotrkubisa/berglas-checksum/runs/4475900810?check_suite_focus=true#step:6:65

github.com/GoogleCloudPlatform/berglas v0.6.2 h1:mgcdWF53ltZA0hBGWrpBAjuKh5VnQPxHRlFRoHd5tpA=
github.com/GoogleCloudPlatform/berglas v0.6.2/go.mod h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI=

With GOPRIVATE=*

• Workflow definition: https://github.com/piotrkubisa/berglas-checksum/blob/master/.github/workflows/with-goprivate.yml#L6-L10
• Test run: https://github.com/piotrkubisa/berglas-checksum/runs/4475900876?check_suite_focus=true#step:6:66

github.com/GoogleCloudPlatform/berglas v0.6.2 h1:GeTSHX7B/eIONuwnVuh92I3f0yAmdSz6RdAYP5/efQI=
github.com/GoogleCloudPlatform/berglas v0.6.2/go.mod h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI=

from berglas.

Okay, that sounds great!

I have strong suspicion towards examples/ directory, which you have named as possible root of the problem. I have done two little changes to the berglas repository in my fork:

• v0.6.3 - rm -rf examples/ completely. The go.sum file started to look exactly the same in both scenarios.
• v0.6.4 - I have reverted changes from v0.6.3 and just renamed the examples dir (mv examples renamed-examples-dir). The workflow says, the checksum is identical in both scenarios too.

$ export GOPRIVATE=*
$ go mod download -json -x github.com/piotrkubisa/[email protected]
{
        "Path": "github.com/piotrkubisa/berglas",
        "Version": "v0.6.4",
        "Info": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.info",
        "GoMod": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.mod",
        "Zip": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.zip",
        "Dir": "/home/piotr/go/pkg/mod/github.com/piotrkubisa/[email protected]",
        "Sum": "h1:d6xGda/YPfMBD3q6Qbpgqi5slRlQ4ULJ5w/8nPW4fK4=",
        "GoModSum": "h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI="
}
$ unset GOPRIVATE
$ go mod download -json -x github.com/piotrkubisa/[email protected]
{
        "Path": "github.com/piotrkubisa/berglas",
        "Version": "v0.6.4",
        "Info": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.info",
        "GoMod": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.mod",
        "Zip": "/home/piotr/go/pkg/mod/cache/download/github.com/piotrkubisa/berglas/@v/v0.6.4.zip",
        "Dir": "/home/piotr/go/pkg/mod/github.com/piotrkubisa/[email protected]",
        "Sum": "h1:d6xGda/YPfMBD3q6Qbpgqi5slRlQ4ULJ5w/8nPW4fK4=",
        "GoModSum": "h1:LEKpytS5wf+P8OGenFlsVmi/uwqcjkFQTH8wZABaCgI="
}

Would the rename from examples/ to demo/ will fix this problem?

from berglas.

This issue is stale because it has been open for 14 days with no
activity. It will automatically close after 7 more days of inactivity.

from berglas.