Might as well use the de-facto standard

shurcooL/go#29

Also add note on what it is downloading, and how it was generated.

The only reason I supported equinox.io is due to the amazing https://github.com/tdewolff/minify

It has since moved to

https://github.com/tdewolff/minify/releases

I'm going to change the sample output to use GitHub.

Looking forward to giving this a try, but it seems busted!

   λ godownloader (master): gi
# github.com/goreleaser/godownloader
./main.go:122:12: project.Build undefined (type *config.Project has no field or method Build)
./main.go:123:10: project.Build undefined (type *config.Project has no field or method Build)
./shell_equinoxio.go:19:9: project.Build undefined (type config.Project has no field or method Build)
./shell_raw.go:32:9: project.Build undefined (type config.Project has no field or method Build)

ok another idea.. maybe not good.

so godownloader got started since I was trying to sync macOS and travis-ci and another linux builder to use the exact same set of tools.

I just realized another way of doing this it to have a special downloader that

1. given command line args, e.g. goinstaller client9/misspell tdewolff/minify ...
2. go to the network and find goreleaser file
3. use that to figure out what to download
4. download it

can be done in parallel.

In other words, instead of godownloader making a shell script, it just actually does the download. And written in go, not gross shell.

For my travis-ci workflow, I would still need to install 'goinstaller' but that would be it.

now i don't care if the authors of a package have provided a godownloader script.

thoughts?

n

interesting.. if there wasn't a goreleaser.yaml file the goinstaller could make a 'best effort' as well.

Should commit dependencies in vendor directory to Git or add directory to .gitignore.

Instead of 404-ing, we should tell the user upfront the platform is not supported, and where to file a bug. We have all the information to do this.

If on Mac, building for say... AWS lambda, you'd like to download the linux/amd64 versions, not the macOS versions.

This for example:

https://github.com/mvdan/sh/releases

just uploads naked golang binaries as releases... not in tar, not in zip.

I've seen this a few times. Anyways experiment.

(BTW, also we also support https://Equinox.io but it's gross as they don't really make it easy to figure out what the last release is).

n

godownloader -repo https://github.com/alecthomas/gometalinter/

That fails, but if I remove the trailing slash it works.

Looks like this line was removed a few days back:

goreleaser/goreleaser@1ed299a#diff-87baee25882e84b0fb93dd0775496706L15

Causing this to no longer work:

https://github.com/goreleaser/godownloader/blob/master/main.go#L38

Not sure what the gameplan is with the defaults, but happy to open a PR if you let me know what needs to be done. Thanks!

Current system requires a GITHUB_TOKEN for use in Travis. Boo 👎

new client9/shlib has a better function now -- no need to use the API at all.

n

Currently, the output is to stdout. However with a timestamp or version info this might create different output every time. This is fine for quick generation, but not good in other cases.

We want a flag to write to file only if the output is significantly different. If the script is different in only comments or a timestamp, then godownloader should not overwrite the file.

Needed for #39
Blocked by #56

if BINDIR is not set, it defaults to ./bin which is maybe not ok.

Show allow flags to change default values.

Add travis badge to readme.md

the checksums file can have another name now...

As per:

golang/go#13560 (comment)

Generated files are marked by a line of text that matches the regular expression, in Go syntax:
^// Code generated .* DO NOT EDIT\.$
The .* means the tool can put whatever folderol it wants in there, but the comment must be a single line and must start with Code generated and end with DO NOT EDIT., with a period.

The text may appear anywhere in the fi

setup travis.ci

Added in

f451dc9#diff-83752e6cf679642f9650d58b739cb2c3

the command for curl added the -N flag

This turns off curl buffering. That's fine, but its not clear what problem this solved. (or maybe I missed it).

I don't see this in similar shell downloaders like:

https://github.com/golang/dep/blob/master/install.sh

Just curious on why, and to make sure it's in older curl versions.

@caarlos0 any ideas here on why it was added?

If it's needed, then I need to update client9/shlibs (no problem, just another thing to do).

n

Thanks again for the tooling!

I tried the following and got:

   λ gh-polls (master): godownloader -repo tj/gh-polls > install.sh
   λ gh-polls (master): sh install.sh
tj/gh-polls: checking GitHub for latest version
install.sh: line 287: adjust_arch: command not found

Much of the code depends on converting uname values back into GOOS and GOARCH values.

However, I really have no idea what uname -m says when running on a arm64 or a ppc64le machine. We have a guess, but it might be wrong.

Make it report an error if we can't convert into a known value.

Just fixed client9/shlib#1 with https://github.com/mvdan/sh shfmt

Really nice. The generated code is good, but with a bit more work it would be completely standardized.

for example

#!/bin/sh
set -e
#  Code generated by godownloader. DO NOT EDIT.
#

Need a timestamp or something here.

This whole thing is one big golang template and we pass in the goreleaser config. To get a timestamp, we'll add a function to the template, and use it.

ala goreleaser/get

Hey thanks for creating this – definitely a time saver!

I'm wondering if you'd be open to adding bash completion scripts. Basically allowing something like this: https://github.com/alecthomas/kingpin#bashzsh-shell-completion to be installed automatically in the install.sh script or if that's already possible

On master, goreleaser allows for multiple build stanzas.

Godownloader works correctly for older yaml files, but not the new format (travis is broken only for new format... old stuff is ok)

For example:

https://github.com/goreleaser/goreleaser/releases/download/v0.17.3/goreleaser_checksums.txt

in

f451dc9#diff-83752e6cf679642f9650d58b739cb2c3

A bunch of status or debug messages were added in shellfn.go such as:

echo "$PREFIX: downloading ${source_url}"

But these aren't in the matching client9/shlib so they'll get over-written when regenerated.

Even better would be a -v or -d flag (verbose, debug) or a -s silent flag in shlibs

n

See #6 and see how

https://github.com/goreleaser/get/blob/master/get

does it.

@caarlos0 @jorinvo --Please excuse my delay, but I'm back to fix this up. I can't find the slack group to chat, so here I am filing a ticket. Any special new rules or processes I should be aware of ?

Using latest bigs and old release of client9/misspell

client9/misspell INFO checking GitHub for latest tag
client9/misspell INFO found version: 0.3.0 for v0.3.0/mac/64bit
client9/misspell DEBUG downloading tarball https://github.com/client9/misspell/releases/download/v0.3.0/misspell_0.3.0_mac_64bit.tar.gz
client9/misspell DEBUG downloading checksum https://github.com/client9/misspell/releases/download/v0.3.0/misspell_0.3.0_checksums.txt
curl: (22) The requested URL returned error: 404 Not Found

turns out the checksum file didn't have the version number it in

https://github.com/client9/misspell/releases/download/v0.3.0/misspell_checksums.txt

Also, ugly CURL error

n

currently, the script doesn't print very much. It should be say what it is doing.

Also:when it does it also doesn't say what script it's from. If multiple scripts are running in parallel, then the output is unclear as to what is happening.

In particular godownloader uses go 1.8

I think it is a good think to add this the usage on the README file. To make user understand faster.

currently requires arg $1 to be 'latest' or a version number.

Hey,

thanks for a great project!

I've noticed that generated script assumes that version number is prefixed with "v". Is there a way to avoid it other than manual changes?

From goreleaser perspective those two formats v0.1.0 and 0.1.0 are equal

Note: we recommend the use of semantic versioning. We are not enforcing it though. We do remove the v prefix and then enforce that the next character is a number. So, v0.1.0 and 0.1.0 are virtually the same and are both accepted, while version0.1.0 is not.

godownloader uses golang native flags
goreleaser uses something else.

They should be the same.

default name of template is in pipeline/defaults/defaults.go:

const NameTemplate = "{{ .Binary }}_{{.Version}}_{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"

But GD isn't using it:

        // TODO: error on conditionals
        if target == "" || target == "{{ .Binary }}_{{ .Os }}_{{ .Arch }}{{ if .
Arm }}v{{ .Arm }}{{ end }}" {
                prefix = "if [ ! -z \"${ARM}\" ]; then ARM=\"v$ARM\"; fi"
                target = "{{ .Binary }}_{{ .Os }}_{{ .Arch }}{{ .Arm }}"
        }

gross.

i.e. goreleaser --make-downloader or something

There are some good reasons for this. It makes the whole vendoring issue for godownloader goes away, less confusing for users, more eyeballs (and hopefully end users), and makes goreleaser that much more complete.

The downside is if we want to add a lot of flags to godownloader to do... I don't know what, it becomes harder (well we could do something like gorelease --make-downloader "option1,option2" or something

I've got a few more days of hacking on godownloader before it makes sense, but something to consider.

thoughts,

n

currently it's in one place at the bottom. but we need to use it in other places.
make it a function

need to support adding headers too.

$> godownloader -repo https://github.com/alecthomas/gometalinter > foo.sh
2018/03/09 16:23:01 Reading https://raw.githubusercontent.com/https://github.com/alecthomas/gometalinter/master/goreleaser.yml
2018/03/09 16:23:01 Issue reading https://raw.githubusercontent.com/https://github.com/alecthomas/gometalinter/master/goreleaser.yml returned: 404 Not Found
2018/03/09 16:23:01 Reading https://raw.githubusercontent.com/https://github.com/alecthomas/gometalinter/master/.goreleaser.yml
2018/03/09 16:23:01  info loading environment variables
2018/03/09 16:23:01  info snapshoting              
2018/03/09 16:23:01  info releasing to GitHub      
2018/03/09 16:23:01  info creating archives        
2018/03/09 16:23:01  info building binaries        
2018/03/09 16:23:01  info creating Linux packages with fpm
2018/03/09 16:23:01  info creating Linux packages with nfpm
2018/03/09 16:23:01  info creating Linux packages with snapcraft
2018/03/09 16:23:01  info calculating checksums    
2018/03/09 16:23:01  info signing artifacts        
2018/03/09 16:23:01  info creating Docker images   
2018/03/09 16:23:01  info releasing to Artifactory 
2018/03/09 16:23:01  info creating homebrew formula
2018/03/09 16:23:01  info creating Scoop Manifest  
$> sh ./foo.sh 
alecthomas/gometalinter info checking GitHub for latest tag
alecthomas/gometalinter info found version: 2.0.5 for v2.0.5/linux/amd64
install: cannot stat '/tmp/tmp.zqpty6BYN2/gometalinter': No such file or directory

The problem is that that after untarring the downloaded tarball in $TMPDIR, it assumes that $TMPDIR contains the binaries. But in this case $TMPDIR contains this:

ls -l /tmp/tmp.zqpty6BYN2/
total 39324
-rw-rw-r-- 1 autarch autarch      312 Mar  9 16:23 gometalinter-2.0.5-checksums.txt
drwxrwxr-x 2 autarch autarch     4096 Mar  9 16:23 gometalinter-2.0.5-linux-amd64
-rw-rw-r-- 1 autarch autarch 40258507 Mar  9 16:23 gometalinter-2.0.5-linux-amd64.tar.gz

Need to adjust shell or curl flags to fail hard if HTTP request isn't a 200.

I can type in bogus stuff, I get "NOT FOUND" from GitHub and script keeps going...

Currently only handles .tar.gz

Similar to #2 and zip file support.

This should be an easy golang template addition

Curious what you think of having a "hosted" version of this. Sort of like https://github.com/tj/bins but still pushing your own build artifacts. That might be a decent compromise. That way you don't have to update ./install.sh scripts often if it's improved for a platform.

Something like:

curl -sf https://bins.io/tj/node-prune | sh

or

curl -sf https://bins.io/github.com/tj/node-prune | sh

versioned

curl -sf https://bins.io/github.com/tj/[email protected] | sh

I'm sure some people would think it's sketchy, but I guess they can manually install :D

we manually deserialize a stream with YAML. We should use the new config.LoadReader

Sad but true, some centos installs don't have which installed

Need to investigate availability of type Or file. And abstract the command away.

Gotta love shell

the samples directory was cool in the beginning but now might be good to remove from .git and allow people to build on demand