version: come up with good default implementation about tailscale HOT 17 CLOSED

If others want to see what's available, you can try:

diff --git a/version/version.go b/version/version.go
index 1bc61d6..3a2562e 100644
--- a/version/version.go
+++ b/version/version.go
@@ -7,5 +7,24 @@
 // Package version provides the version that the binary was built at.
 package version
 
-const LONG = "LONGVER-TODO"
-const SHORT = "SHORTVER-TODO"
+import (
+       "fmt"
+       "os"
+
+       "rsc.io/goversion/version"
+)
+
+var LONG = "VERSION-UNKNOWN"
+var SHORT = "VERSION-UNKNOWN"
+
+func init() {
+       n, err := os.Executable()
+       if err != nil {
+               return
+       }
+       v, err := version.ReadExe(n)
+       if err != nil {
+               println(err.Error())
+       }
+       println(fmt.Sprintf("Version: %+v\n", v))
+}

Then go run ./cmd/tailscaled -h.

from tailscale.

I'm getting an increasing number of inquiries about this in the context of OSS packaging in distros. It's a problem that we can't ship sensible version info if you're building from just the OSS repo.

I'm going to take a swing at fixing this ahead of the Go compiler providing us more useful information, at least well enough that distro packagers can embed version info as they build, while we can continue to provide corp version info for the non-free programs.

from tailscale.

What's your plan?

from tailscale.

Rough plan, not tested out yet:

• Make a richer version type that embeds more information (short version per git describe, git commit hash, and an "is OSS version" bool that means "do the other fields refer to the OSS repo or the corp one?").
• Add a shell script at the root of the OSS repo that does what redo currently does in our corp repo - do a bunch of git introspection to generate version data. Distro packagers can be given documentation to run that when they build packages (and we can make the CLI complain at people when version info is missing). Doesn't solve the case of an individual building by themselves with go get, but I'm okay waiting on Go for that case. We can continue to do some ratcheting date-based fallback for that case if we really want to.
• Add tailscale.com/version.Set(v), to allow our non-oss binaries to override the OSS build info with their own version object. This lets the Windows and Apple builds provide build info from the corp repo, while still letting all our code uniformly use tailscale.com/version for version information.
• Our corp repo continues to use redo to generate its own private version info, and our non-oss binaries call version.Set early in execution.

Works in the following use cases:

• Us building packages from the corp repo works, because oss/cmd/* won't override the OSS version with the corp version info. Need a tiny bit of adjustment so that redo regenerates oss version info, trivial. Slight risk of version confusion over wireguard-go, since we do go.mod replace shenanigans, but in practice we keep the go.mod version and our replaced version in sync anyway, so should be fine.
• OSS distro packagers (e.g. Arch, Nix) already do complicated non-go-get type build processes, it's trivial for them to insert ./generate-version.sh before the compilation phase to get version info that exactly matches what we embed in packages we make.
• Us building the non-oss binaries (Windows, macOS, iOS) still works, because we have a corp version stamp and corp/cmd/* binaries override the OSS stamp with the corp one at startup.
• Richer version info means we can format versions like oss-0.96-123 or corp-0.96-123 (pls bikeshed exact format in PR later :) ), and avoid descriptor confusion when we're staring at stats. Having this data in a structured type means we can also build tooling that doesn't have to parse version strings.

That's the general shower thoughts I had this morning. The context is that I reviewed the initial Nix/NixOS derivations for Tailscale last night, which are going to ship to NixOS users with the crappy fallbackversion info. Separately this morning, an Arch user pointed out that some of my AUR packages violate AUR convention by shipping binaries, and I need to switch to building from source - which would force me to implement some "version for OSS" hack anyway. Given those, I want to solve that problem ONCE AND FOR ALL upstream, so I can give downstream distro packagers uniform instructions for how to build binaries with good version data.

from tailscale.

So, another bespoke Makefile / make.bash / make.go for oss. (golang/go#37475)

That's fine for now, for people who want to use it.

I have a crazier idea for the go get users: look up the version from a network service at runtime based on the program taking a signature of its own binary (using debug/elf, etc). The network service would serve effectively a map[Signature]struct{GitCommit, CommitTime} that would be populated by a daemon repeatedly building each commit with a dozen latest/most populate versions of the compiler. I might even just build that as a non-Tailscale service on Cloud Run that anybody could use for free.

from tailscale.

Pretty much, yeah. It's definitely not what I want to do, but the third-party distros need something to embed version information we can reason about for debugging.

A versioning service sounds interesting, though I question whether you can actually account for the variety of weirdery out there (e.g. all Nix binaries have altered library load paths and whatnots, so the compilers are likely to produce slightly different output). I think it's possible, but it'd be fiddly.

from tailscale.

Yo,

This issue has been silent since March and assigned a low priority. But with the release of version 1.0 I think you need to reconsider.

Today if I want to download tailscale there are a few binaries I can select. But no signatures. That isn't a problem in theory, go binaries are reproducible. However there is no clear way to figure out how they are built. Looking at the built version

λ tailscale_1.0.5_amd64 » ./tailscale --version          
1.0.5-g31b5dec0a

I can checkout the 1.0.5 tag, and try build it with -trimpath. But the commit g31b5dec0a is not present in this repository. Even injecting the correct version into the binary gives me a different checksum of the binaries.

What are the major roadblocks for embedding version information, and streamlining the build process? It would help reproducible builds for your project and distro packagers (semi-relevant as I'm interested distributing tailscale in the [community] repo of Arch).

from tailscale.

This got blocked on golang/go#37475 which is accepted but not yet implemented in upstream Go.

Our -g31b5dec0a suffix means "git commit" 31b5dec0a, but that's a git commit of our parent repo that has all our other repos (e.g. https://github.com/tailscale/tailscale-android, the closed-source iOS app) as git submodules.

I would suggest solutions but I'm not sure your goal. You mentioned several somewhat separate concerns.

Our builds don't contain anything closed-source mixed in from the parent repo other than the version string, so if you set that, use -trimpath, and build with the same version of Go that we do (https://github.com/tailscale/go, the commit of which we print out at start-up), then you should get the same output bytes.

For Arch builds, I'd just make the version be "1.0.5-g<git commit of tailscale/tailscale repo>"

from tailscale.

I would suggest solutions but I'm not sure your goal. You mentioned several somewhat separate concerns.

My goal is to have tailscale reproducible. This is important both when distributing binaries with no signatures, and for packagers to have a streamlined build process. My impression was that this versioning problem was the reason why I can't have a make build in the project. But it seems like I was mistaken.

Our builds don't contain anything closed-source mixed in from the parent repo other than the version string, so if you set that, use -trimpath, and build with the same version of Go that we do (https://github.com/tailscale/go, the commit of which we print out at start-up), then you should get the same output bytes.

So I tried. I fetched https://github.com/tailscale/go/releases/tag/build-56db76510f9640d2ad652f206ae6e41c1a5d63ca which is what 1.0.5-g31b5dec0a should be built with (output of running tailscale). Ran ./src/make.bash in the go source. Tried building tailscale by inserting the correct version and running the following compilation options which is purely guessed as it's not dynamically compiled.

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 ~/Downloads/go/bin/go build -trimpath ./cmd/tailscale

λ ~ » sha256sum Downloads/tailscale_1.0.5_amd64/tailscale 
05ce75560631d6723a73fdac7a6711f8d5e142dad8bae7c2b14f8c13c61077b6  Downloads/tailscale_1.0.5_amd64/tailscale
λ ~ » sha256sum Git/prosjekter/Go/tailscale/tailscale 
4791c43c5f1f24f3177731cd93bbd0f56d764b03a27fea919a0465acfa9a6326  Git/prosjekter/Go/tailscale/tailscale
λ ~ » ./Git/prosjekter/Go/tailscale/tailscale --version
1.0.5-g31b5dec0a
λ ~ » ./Downloads/tailscale_1.0.5_amd64/tailscale --version
1.0.5-g31b5dec0a

I don't know how the bootstrapping process affects the results, and I'm unsure if it's feasibly without you redistributing the compiler itself (what guarantees does the go compiler have?). Please do point out if I have made any mistakes with this, I'm just guessing my way around at this point.

I think this is a problem, but I'm unsure if it's related to this issue specifically. Currently there doesn't seem to be a good way to prove how the distributed tailscale is being built.

from tailscale.

from tailscale.

Oh, we also set the tailscale_go build tag in our builds.

Relevant bits from our build system:

redo  out/x86_64-linux/oss/cmd/tailscale/tailscale (resumed)
#!/bin/sh -e
# Per-platform settings for the go compiler.
# Auto-generated by go.od. Do not edit!
S='/home/bradfitz/src/tailscale.io'
export GO='/home/bradfitz/.cache/tailscale-go/bin/go'
export CC='cc'
export CFLAGS=''
export GO_LDFLAGS='-extldflags ""'
export GOOS=''
export GOARCH=''
export GOARM=''
export CGO_ENABLED='0'

# Necessary because CI forces GOROOT and we need
# to override it back.
export GOROOT=
redo    (../../../../../oss/version/all)                                                                                                                                            
redo    (../../../../../oss/version/version.h)
redo    oss/version/describe.txt
redo    oss/version/describe.txt (done)                                                                                                                                             

redo  out/x86_64-linux/oss/cmd/tailscale/tailscale (done)                                                                                                                           

....

        '$GO' build \\
            -v -trimpath -tags=redo,tailscale_go \\
            -ldflags "\$GO_LDFLAGS" \\
            -o "\$t" \\
            "\$pkg"

from tailscale.

(This is all pretty off topic for this issue, though.)

from tailscale.

(Thanks,!I'm aware it's offtopic. I'm happy to take up this discussion wherever appropriate.)

from tailscale.

If you like Slack, see https://github.com/tailscale/tailscale/wiki/Community for a link. Otherwise if you prefer GitHub, just create a new issue here? Perhaps titled "Document how to generate the same binaries Tailscale distributes"

from tailscale.

I think the main intent of this issue has been addressed by https://github.com/tailscale/tailscale/blob/main/build_dist.sh
Specific questions about reproducibility split out into #779.

from tailscale.

https://go-review.googlesource.com/c/go/+/353930 will embed the git hash into generated binaries. I think that will be sufficient to cover the remaining case of concern in this issue, of people building from source.

from tailscale.

Waiting on Go 1.18 in couple weeks.

from tailscale.