strangelove-ventures / heighliner Goto Github PK

View Code? Open in Web Editor NEW

48.0 48.0 39.0 582 KB

Repository of docker images for the node software of Cosmos chains

License: Apache License 2.0

Dockerfile 54.45% Go 45.55%

heighliner's People

Contributors

Stargazers

Watchers

Forkers

kinrokinro arirubinstein jasonlutz-chainguard blasrodri misko9 oshorefueled gle1pn1r uwezukwechibuzor bigs lockwarr polymerdao aljo242 consideritdone gbuchukuri discoverdefiteam kyvenetwork hard-nett cbarraford nextnet-works flaners bharvest-devops udpatil teddyknox iramiller spoo-bar pfc-developer kakucodes kylemoser pwnfoo yerasyla blacksandtech qj0r9j0vc2 terra-money moonyongjung toliaqat agoric-labs noble-assets

heighliner's Issues

Heighliner should include Bandchain in its list of packages.

chains.yaml:

# Bandchain
- name: bandchain
  github-organization: bandprotocol
  github-repo: chain
  language: go
  build-target: install
  binaries:
    - /go/bin/bandd
  build-env:
    - BUILD_TAGS=muslc

Error:

...
# github.com/bandprotocol/chain/v2/cmd/bandd
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: librt.so.1, needed by /go/pkg/mod/github.com/bandprotocol/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libpthread.so.0, needed by /go/pkg/mod/github.com/bandprotocol/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libm.so.6, needed by /go/pkg/mod/github.com/bandprotocol/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libdl.so.2, needed by /go/pkg/mod/github.com/bandprotocol/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libc.so.6, needed by /go/pkg/mod/github.com/bandprotocol/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
...

Penumbra pcli failure

While Heighliner image builds successfully, pcli binary, does not run correctly in image.

This issue begins on penumbra release tag: 038-kalyke.
On older tags, pcli works fine.

(I'm using an m1 mac)

Remove Replace Directive in go.mod to allow global installs using `go install`

I came across this while building a CircleCI workflow for building Duality. Heighliner is not a one-line install because of the replace directive here: https://github.com/strangelove-ventures/heighliner/blob/main/go.mod#L85

I'm not sure what it's for but if it were removed it would resolve this error:

➜  go install github.com/strangelove-ventures/heighliner@latest
go: github.com/strangelove-ventures/heighliner@latest (in github.com/strangelove-ventures/[email protected]):
	The go.mod file for the module providing named packages contains one or
	more replace directives. It must not contain directives that would cause
	it to be interpreted differently than if it were the main module.

osmosis - v20.2.1 - eip1559state.json :-(

looks like osmosis v20.2.1 introduced a new feature (eip1559) which is attempting to write a file into the config/ directory.
(eip1559state.json).

I don't believe there is a config directive on where to store this. the sentry does start, but I assume it isn't performing as designed.

just a heads up for now.. hoping they will provide a config directive to put this into a /tmp/ directory or somesuch

non wasm chains always seem to fail on build w/ standard simapp

if wasmvm is not found in go.mod or go.sum, I constantly get wasm errors. But not for anything using wasm with the local docker. If I just add wasmvm 1.5.2 to the go.mod/go.sum (without any app usage) it builds fine since it can download.

- name: minimalpoa
  dockerfile: cosmos
  build-target: make install
  binaries:
    - /go/bin/aaaaad
  build-env:
    - LEDGER_ENABLED=false

It's possible it just needs a build-env or something. But feel this was meant to be automatic

GH Action logs

#11 [go-builder 3/7] WORKDIR /code
#11 DONE 0.0s
#12 [go-builder 4/7] ADD go.mod go.sum ./
#12 DONE 0.0s
#13 [go-builder 5/7] RUN set -eux;     export ARCH=$(uname -m);     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}');     if [ ! -z "${WASM_VERSION}" ]; then       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;)     fi;     go mod download;
#13 0.051 + set -eux
#13 0.051 + uname -m
#13 0.052 + export 'ARCH=x86_64'
#13 0.053 + go list -m all
#13 0.055 + grep github.com/CosmWasm/wasmvm
#13 0.057 + awk '{print $2}'
#13 8.725 + WASM_VERSION=
#13 ERROR: process "/bin/sh -ecuxo pipefail set -eux;     export ARCH=$(uname -m);     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}');     if [ ! -z \"${WASM_VERSION}\" ]; then       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;)     fi;     go mod download;" did not complete successfully: exit code: 1
------
 > [go-builder 5/7] RUN set -eux;     export ARCH=$(uname -m);     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}');     if [ ! -z "${WASM_VERSION}" ]; then       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;)     fi;     go mod download;:
0.051 + set -eux
0.051 + uname -m
0.052 + export 'ARCH=x86_64'
0.053 + go list -m all
0.055 + grep github.com/CosmWasm/wasmvm
0.057 + awk '{print $2}'
------
Dockerfile:10
--------------------
   9 |     ADD go.mod go.sum ./
  10 | >>> RUN set -eux; \
  11 | >>>     export ARCH=$(uname -m); \
  12 | >>>     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}'); \
  13 | >>>     if [ ! -z "${WASM_VERSION}" ]; then \
  14 | >>>       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;) \
  15 | >>>     fi; \
  16 | >>>     go mod download;
  17 |     
--------------------
ERROR: failed to solve: process "/bin/sh -ecuxo pipefail set -eux;     export ARCH=$(uname -m);     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}');     if [ ! -z \"${WASM_VERSION}\" ]; then       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;)     fi;     go mod download;" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -ecuxo pipefail set -eux;     export ARCH=$(uname -m);     WASM_VERSION=$(go list -m all | grep github.com/CosmWasm/wasmvm | awk '{print $2}');     if [ ! -z \"${WASM_VERSION}\" ]; then       wget -O /lib/libwasmvm_muslc.a [https://github.com/CosmWasm/wasmvm/releases/download/${WASM_VERSION}/libwasmvm_muslc.${ARCH}.a;](https://github.com/CosmWasm/wasmvm/releases/download/$%7BWASM_VERSION%7D/libwasmvm_muslc.$%7BARCH%7D.a;)     fi;     go mod download;" did not complete successfully: exit code: 1

Osmosis fails to build

To reproduce:

 heighliner build -b --buildkit-addr tcp://127.0.0.1:8125 -r ghcr.io/strangelove-ventures/heighliner -c osmosis -v v12.2.0

Error:

 => => # cp: can't stat 'build/osmosisd': No such file or directory
 => [linux/amd64->arm64 build-env 14/18] RUN bash -c   'BINARIES_ARR=($BINARIES_ENV);   cd /root/bin ;   for BINARY in "${BINARIES_ARR[@]}"; do     BINSPLIT=(${BINARY//:/ }) ;     BINPATH=${BINSPLIT[1]} ;     if [ ! -z "$BINPATH" ]; then       mkdir -p "$(dirname "${BINPATH}")" ;       cp ${BINSPLIT[0]} "${BINPATH}";     else       cp ${BINSPLIT[0]} . ;    1.5s
 => => # cp: can't stat 'build/osmosisd': No such file or directory

Test harness

Build test harness with coverage of all supported scenarios in README, and have it run as part of CI

Heighliner should have compatiblity with Ignite client builds.

There should be an additional language: ignite

Heighliner should include Odin in its list of packages.

chain.yaml:

# Odin
- name: odin
  github-organization: ODIN-PROTOCOL
  github-repo: odin-core
  language: go
  build-target: install
  binaries:
    - /go/bin/odind

Error:

...
# github.com/ODIN-PROTOCOL/odin-core/cmd/odind
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: librt.so.1, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libpthread.so.0, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libm.so.6, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libdl.so.2, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libc.so.6, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: ld-linux-x86-64.so.2, needed by /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so: undefined reference to `memset@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/slandymani/[email protected]/api/libgo_owasm.so: undefined reference to `pthread_cond_signal@GLIBC_2.3.2'
...

Fix agoric build as of agoric-upgrade-11-rc0

Getting error:

heighliner build -r ghcr.io/strangelove-ventures/heighliner -c agoric -g agoric-upgrade-11-rc0 -s
Loaded chains from /home/danb/code/github/strangelove-ventures/heighliner/chains.yaml
Using local cargo/native.Dockerfile
error building docker image for agoric from ref: agoric-upgrade-11-rc0 - error getting mod file: failed to open go.mod file: file does not exist

Build github actions workflow

Build github actions workflow so that heighliner images can be built and push from an importable workflow.

Examples

With built-in chain config

      - name: Build and push heighliner Docker image
        uses: heighliner/build-push-action@v1
        with:
          chain: gaia

Custom chain config

      - name: Build and push heighliner Docker image
        uses: heighliner/build-push-action@v1
        with:
          chain: somechain
          dockerfile: cosmos
          build-target: install
          build-env:
            - LEDGER_ENABLED=false
            - BUILD_TAGS=muslc
          binaries:
            - /go/bin/somechaind

injective woes

couple of minor issues with injective build.

injective-exchange isn't present in their most recent zip files.
peggo fails to run due to certificate issues
jq isn't present (it is in other heighliners)

my solution to this was to modify none/Dockerfile (for point 2 & 3)

diff --git a/dockerfile/none/Dockerfile b/dockerfile/none/Dockerfile
index ad520c5..53172c2 100644
--- a/dockerfile/none/Dockerfile
+++ b/dockerfile/none/Dockerfile
@@ -37,7 +37,9 @@ FROM debian:bullseye
 # FROM ubuntu:22.04

 LABEL org.opencontainers.image.source="https://github.com/strangelove-ventures/heighliner"
-
+RUN apt update -y
+RUN apt install -y jq ca-certificates curl
+RUN /usr/sbin/update-ca-certificates
 # Install binaries
 COPY --from=build-env /root/bin /usr/bin

ghcr containter for Terp Network does not exist

hey there!

Looking for some clarity on how to include Terp Network into your interchaintest repo.

What would be needed for this to happen? Still learning ghcr, github workflows, and the interchaintest repo itself so please excuse me!!

Cheers

Add All Chains From Chain-Reg

Add arm64 images

Helpful for development on m1 macs and deployment to hardware such as AWS Graviton

Building Cargo images using a remote buildkit server fails

When building a Cargo image from my local machine using a remote buildkit server, I get this error:
failed to solve: failed to fetch oauth token: unexpected status: 403 Forbidden

Screenshot:

Onomy artifact does not work

 go run ./main.go build --chain onomy --registry ghcr.io/strangelove-ventures/heighliner --version v1.0.0

Result:

❯ docker run --entrypoint sh -it ghcr.io/strangelove-ventures/heighliner/onomy:v1.0.0
~ $ onomyd
sh: onomyd: not found
~ $ exit

As a workaround, I'm going to create and push my own docker image.

gaia v8 fails to build due to go version enforcement

#24 3.858 + sh -c 'make install'
#24 3.877 ERROR: Go version 1.18 is required for v8.0.0 of Gaia.

For cosmos dockerfiles, heighliner should have a mechanism to determine the go version from go.mod of the chain source, and use that go version to build.

Axelar nodejs version mismatch

I think v16.0.0 is what's required.

cd axelar-cgp-solidity
+ npm ci
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@axelar-network/[email protected]',
npm WARN EBADENGINE   required: { node: '>=16 <17' },
npm WARN EBADENGINE   current: { node: 'v18.14.1', npm: '9.1.2' }
npm WARN EBADENGINE }
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '^12.0.0 || ^14.0.0 || ^16.0.0' },
npm WARN EBADENGINE   current: { node: 'v18.14.1', npm: '9.1.2' }
npm WARN EBADENGINE }

 @axelar-network/[email protected] flatten
> sh scripts/flatten-contracts.sh

Flattening contracts...
You are using a version of Node.js that is not supported by Hardhat, and it may work incorrectly, or not work at all.

Please, make sure you are using a supported version of Node.js.

It still builds but the warnings seems risky.

Getting error when running e2e test for async-icq function

I am building image of cosmos chain with following config in chains.yml. With no custom docker file.

- name: icqd
  github-organization: aerius-labs
  github-repo: athena
  dockerfile: cosmos
  build-target: go build -ldflags "$LDFLAGS" -o build/icq ./cmd/athenad
  binaries:
    - build/icq

I got the following errors when I am run the e2e tests.

  logger.go:130: 2023-09-12T15:09:45.425+0530    ERROR    Failed to pull image    {"error": "invalid reference format", "repository": "ghcr.io/suniljalandhra/athena:latest", "tag": "latest"}
    logger.go:130: 2023-09-12T15:09:45.425+0530    ERROR    Failed to pull image    {"error": "invalid reference format", "repository": "ghcr.io/suniljalandhra/athena:latest", "tag": "latest"}
    icq_test.go:111:
            Error Trace:    /Users/suniljalandhra/Desktop/aerius/athena/x/qcrescent/e2e/icq_test.go:111
            Error:          Received unexpected error:
                            failed to create faucet accounts: failed to create common account with name faucet: failed to create key with name "faucet" on chain controller: image ghcr.io/suniljalandhra/athena:latest:latest: pull image ghcr.io/suniljalandhra/athena:latest:latest: invalid reference format
            Test:           TestInterchainQueries

Build: Oraichain

Chain.yaml:

# Oraichain
- name: oraichain
  github-organization: oraichain
  github-repo: orai
  language: go
  build-target: build
  binaries:
    - /go/bin/oraid
  build-env:
    - BUILD_TAGS=muslc
  pre-build: |
    go get github.com/CosmWasm/wasmd
    go mod tidy

Error:

BUILD_TAGS=muslc make go-build
make[1]: Entering directory '/go/src/github.com/oraichain/orai'
--> Ensure dependencies have not been modified
all modules verified
go build -mod=readonly -tags "netgo,muslc" -ldflags '-X github.com/cosmos/cosmos-sdk/version.Name=orai -X github.com/cosmos/cosmos-sdk/version.AppName=oraid -X github.com/cosmos/cosmos-sdk/version.Version=v0.40.4_ibc -X github.com/cosmos/cosmos-sdk/version.Commit=a963e77d85b89dc8d5c5a7edc16ff7ceb5a34206 -X github.com/oraichain/orai/app.Bech32Prefix=orai -X "github.com/cosmos/cosmos-sdk/version.BuildTags=netgo,muslc"' -trimpath -o build/oraid ./cmd/oraid
# github.com/CosmWasm/wasmvm/api
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_C2func_create':
/tmp/go-build/cgo-gcc-prolog:56: undefined reference to `create'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_C2func_get_code':
/tmp/go-build/cgo-gcc-prolog:80: undefined reference to `get_code'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_C2func_handle':
/tmp/go-build/cgo-gcc-prolog:115: undefined reference to `handle'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_Cfunc_create':
/tmp/go-build/cgo-gcc-prolog:267: undefined reference to `create'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_Cfunc_get_code':
/tmp/go-build/cgo-gcc-prolog:287: undefined reference to `get_code'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x004.o: in function `_cgo_72d041672a77_Cfunc_handle':
/tmp/go-build/cgo-gcc-prolog:318: undefined reference to `handle'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x006.o: in function `_cgo_72d041672a77_Cfunc_allocate_rust':
/tmp/go-build/cgo-gcc-prolog:53: undefined reference to `allocate_rust'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: $WORK/b062/_x006.o: in function `_cgo_72d041672a77_Cfunc_free_rust':
/tmp/go-build/cgo-gcc-prolog:80: undefined reference to `free_rust'
collect2: error: ld returned 1 exit status
make[1]: Leaving directory '/go/src/github.com/oraichain/orai'
make[1]: *** [Makefile:84: go-build] Error 2
make: *** [Makefile:77: build] Error 2
2022/08/08 16:01:03 error building docker image for oraichain:v0.40.4-ibc-beta - The command '/bin/sh -c [ ! -z "$PRE_BUILD" ] && sh -c "${PRE_BUILD}";     [ ! -z "$BUILD_ENV" ] && export ${BUILD_ENV};     [ ! -z "$BUILD_TAGS" ] && export "${BUILD_TAGS}";     [ ! -z "$BUILD_DIR" ] && cd "${BUILD_DIR}";     make ${BUILD_TARGET}' returned a non-zero code: 2

panic: Some images failed to build

CMD:
./heighliner build -c oraichain -v v0.40.4-ibc-beta

Move `ADD` later to take advantage of cache

When running heighliner build -c juno --local -f ./chains.yaml the CMD ADD . . is at the top of the file. Meaning any small change requires an entire re-compile of every base dependency non chain related as well. (~35 minutes)

This should be moved to as late as possible before compiling to optimize speeds

Add Celestia

Add default `GORACE` environment variable to race-detector enabled builds

Per https://go.dev/doc/articles/race_detector, by default, a race-detector-enabled build will print data races to stderr and continue execution.

IMO, if someone is using the -race tagged builds, they are looking for fast feedback on the first data race encountered. Setting GORACE=halt_on_error=1 will cause the program to exit with code 66 on the first data race encountered, whereas the default value will continue execution to the end and probably exit 0.

I think the feedback would be clearer if defaulting to halt on error, and the user is always free to override it back to GORACE=halt_on_error=0 if they do want execution to continue past the first encountered data race.

bug: when building the image by passing all overrides, not able to provide image name

Heighliner commit hash: 2da0b66a365ef9dfd58230f137473b1b264a4927

Steps to reproduce:

Trying to build an Heighliner image of a chain which isn't part of the ./chains.yaml file by providing overrides.

heighliner build --org archway-network --repo archway --dockerfile cosmos --build-target "make build" --build-env "BUILD_TAGS=muslc" --binaries "build/archwayd" --git-ref v3.0.0 --tag test

Alternatively, also tried the below command but still didn't work

heighliner build archway --org archway-network --repo archway --dockerfile cosmos --build-target "make build" --build-env "BUILD_TAGS=muslc" --binaries "build/archwayd" --git-ref v3.0.0 --tag test

Actual behavior:

The generated image name is acrechain:test, which is just the name of the first chain in the chains.yaml.

Expected behavior:

Generate an image with the name archway:test.

It would be quite useful to be able to provide custom names for the generated images.

Relevant Trace:

Loaded chains from /mnt/c/code/heighliner/chains.yaml
Using local cosmos/native.Dockerfile
Go version from go.mod: 1.19, will build with version: 1.19.6 image: 1.19.6-alpine3.17
Building image from ref: v3.0.0, resulting docker image tags: +[acrechain:test]

CICD should lint the `chains.yaml` and fail if not valid

If the chains.yaml is malformed, heighliner will not work at all.

Axelar fails to build due to new golangci-lint prereq

This happens in make generate which calls make prereqs.

missing go.sum entry for module providing package github.com/golangci/golangci-lint/cmd/golangci-lint; to add:
        go mod download github.com/golangci/golangci-lint
make: *** [Makefile:116: prereqs] Error 1

I've tried several ways to remove this error including the suggested recovery command go mod download. I still get the same error.

Added to makefile in revision afdb326815b5382e84f14782afa4d34c085d5048 on 2/15/2023.

The last line is problem, where they added golangci-lint.

# Install all generate prerequisites
.Phony: prereqs
prereqs:
	@which goimports &>/dev/null	 ||	go install golang.org/x/tools/cmd/goimports
	@which moq &>/dev/null			 ||	go install github.com/matryer/moq
	@which statik &>/dev/null        ||	go install github.com/rakyll/statik
	@which mdformat &>/dev/null 	 ||	pip3 install mdformat
	@which protoc &>/dev/null 		 || 	echo "Please install protoc for grpc (https://grpc.io/docs/languages/go/quickstart/)"
	@which golangci-lint &>/dev/null ||  go install github.com/golangci/golangci-lint/cmd/golangci-lint

Currently, axelar builds will fail without manual intervention.

Add Juno MEV (skip protocol) to chains.yaml

Prevents us from needing to manually build the image on each upgrade.

Axelar switched to go1.19

@validator Hey folks, we have switched to building axelard with go 1.19. Please update your build process to use that if you aren't already on it. You can test it out on the main branch: https://github.com/axelarnetwork/axelar-core
The upcoming release (tentatively planned for Tuesday, more details to come) will be using go 1.19

https://discord.com/channels/770814806105128977/869603957360713778/1075560955699810324

Heighliner should report `version` info.

When I heighliner version --long
Then I should see:

The project name
Server name
Version number
Commit hash
Build tags
Go version
List of build dependencies with versions

Title

Add Version Support for Heighliner

Description

Request to add version support to Heighliner to enhance user experience and supportability. This would allow users to quickly verify the version, commit hash, build tags, Go version, and dependencies used to build the heighliner binary.

Proposed Feature Details

Implement a new command heighliner version --long that outputs detailed version information, similar to the provided example. This should include:

The project name
Server name
Version number
Commit hash
Build tags
Go version
List of build dependencies with versions

Example Output

heighliner version --long
name: heighliner
server_name: heighlinerd
version: x.y.z
commit: <commit_hash>
build_tags: <tags>,
go: go version go1.x.x <platform>
build_deps:
- dependency1@version
- dependency2@version
...

This feature will assist in troubleshooting, enhance the development workflow, and ensure compatibility across different environments.

To typically add version support in a Go project, you would:

Define a Version Package: Create a package dedicated to managing version information. This package can contain variables for the version number, commit hash, build date, and other relevant information.
Populate Version Information at Build Time: Use Go's -ldflags option to inject version information at build time. This can be done within a build script or your CI/CD pipeline.
```
go build -ldflags "-X main.version=1.0.0 -X main.commit=$(git rev-parse HEAD) -X main.date=$(date +%F)"
```
Implement the Version Command: In your application's command-line interface (usually using a package like cobra for Go CLI applications), implement a version command that outputs this information.
Retrieve and Display Version Information: Within the version command, retrieve the version information from the version package and display it to the user.

Example version.go in the version package might look like this:

package version

var (
    Version   = "dev" // overridden at build time
    Commit    = "none" // overridden at build time
    BuildDate = "unknown" // overridden at build time
)

And in your CLI handling part:

import (
    "fmt"
    "yourproject/version"
)

func printVersion() {
    fmt.Printf("Version: %s\nCommit: %s\nBuild Date: %s\n", version.Version, version.Commit, version.BuildDate)
}

Remember to replace "yourproject/version" with the actual import path of your version package.

Option: CLI options when no chains.yaml file exists?

I have noticed some software allows you to pass all options as cli arguments instead of using a conf file. Is this something we want to consider for heighliner?

support `--build` arguments

It would be nice if we could do something like this..excuse the if logic

add ability to specify go-version in chains.yaml

currently the code determines the go version via parsing the go.mod file, but some chains (kujira) do version checking in their makefile.

this causes heighliner to use the 1.18 alpine version (and fails) instead of the 1.19 one.

and yes.. i've asked kujira to fix their repo, but it would also be good if this tool could build regardless.

Heighliner hardcodes wasm lib version

While using the latest version of Juno in IbcTest, the nodes failed to start with the following log:
Error: libwasmversion mismatch. got: 1.0.0; expected: v1.1.1

If the chain uses wasm, heighliner should query and download the correct version of wasmvm.

cheqd artifact does not work

The cheqd binary seems to be have been built incorrectly.

Inside the cheqd:v1.4.4 docker

~ $ which cheqd-noded
/bin/cheqd-noded
~ $ /bin/cheqd-noded
sh: /bin/cheqd-noded: not found

I looked at a similar issue but the Makefile already consumes env LDFLAGS

Help would be appreciated! thanks!

Agoric fails to build

Command from our heighliner build server:

heighliner build -b --buildkit-addr tcp://127.0.0.1:8125 -r ghcr.io/strangelove-ventures/heighliner -c agoric -v pismoD

Error:

 => ERROR [linux/amd64->arm64 build-env  9/13] RUN set -eux;    if [ ! -z "1.18.10" ]; then export PATH=/usr/local/cargo/bin:/usr/local/sbin:/usr/local/b  119.2s
 => CACHED [linux/amd64 build-env  5/13] RUN git clone -b pismoD --single-branch https://github.com/Agoric/agoric-sdk.git --recursive                        0.0s
 => CACHED [linux/amd64 build-env  6/13] WORKDIR /build/agoric-sdk                                                                                           0.0s
 => CACHED [linux/amd64 build-env  7/13] RUN if [ ! -z "" ]; then      if [ ! -z "" ]; then cd ""; fi;      if [ ! -f "Cargo.toml" ]; then exit 0; fi;       0.0s
 => CACHED [linux/amd64 build-env  8/13] RUN set -eux;     if [ ! -z "1.18.10" ]; then      wget https://go.dev/dl/go1.18.10.linux-amd64.tar.gz  -O - | tar  0.0s
 => [linux/amd64 build-env  9/13] RUN set -eux;    if [ ! -z "1.18.10" ]; then export PATH=/usr/local/cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:  119.4s
 => => # 7771c926464a98b0a902087fdc1bddabd5
 => => # + go build -v -mod=readonly -tags BUILD_TAGS=muslc -ldflags -X github.com/cosmos/cosmos-sdk/version.Name=agoric   -X github.com/cosmos/cosmos-sdk/versio
 => => # n.AppName=agd   -X github.com/cosmos/cosmos-sdk/version.Version=pismoD   -X github.com/cosmos/cosmos-sdk/version.BuildTags=BUILD_TAGS=muslc   -X github.
 => => # com/cosmos/cosmos-sdk/version.Commit=f760927771c926464a98b0a902087fdc1bddabd5 -buildmode=c-shared -o build/libagcosmosdaemon.so ./cmd/libdaemon/main.go
 => => # go: downloading github.com/agoric-labs/tendermint v0.34.23-alpha.agoric.3
 => => # go: downloading github.com/agoric-labs/cosmos-sdk v0.45.11-alpha.agoric.1.1
error building docker image for agoric from ref: pismoD - failed to solve: process "/bin/sh -c set -eux;    if [ ! -z \"$GO_VERSION\" ]; then export PATH=$PATH:/usr/local/go/bin; fi;    if [ \"$TARGETARCH\" = \"arm64\" ]; then export ARCH=aarch64 CAPS=AARCH64;    elif [ \"$TARGETARCH\" = \"amd64\" ]; then export ARCH=x86_64 CAPS=x86_64; fi;    export CARGO_BUILD_TARGET=${ARCH}-unknown-linux-gnu;    if [ \"$TARGETARCH\" != \"$BUILDARCH\" ]; then      export CARGO_TARGET_${CAPS}_UNKNOWN_LINUX_GNU_LINKER=${ARCH}-linux-gnu-gcc        CC_${ARCH}_unknown_linux_gnu=${ARCH}-linux-gnu-gcc        CXX_${ARCH}_unknown_linux_gnu=${ARCH}-linux-gnu-g++        PKG_CONFIG_SYSROOT_DIR=/usr/${ARCH}-linux-gnu;    fi;    [ ! -z \"$PRE_BUILD\" ] && sh -c \"${PRE_BUILD}\";    if [ ! -z \"$BUILD_TARGET\" ]; then      if [ ! -z \"$BUILD_ENV\" ]; then export ${BUILD_ENV}; fi;      if [ ! -z \"$BUILD_TAGS\" ]; then export \"${BUILD_TAGS}\"; fi;      if [ ! -z \"$BUILD_DIR\" ]; then cd \"${BUILD_DIR}\"; fi;      sh -c \"${BUILD_TARGET}\";    fi" did not complete successfully: exit code: 1

panic: Some images failed to build

goroutine 1 [running]:
github.com/strangelove-ventures/heighliner/builder.(*HeighlinerBuilder).BuildImages(0xc0003acfa0)
        /home/strangelove/heighliner/builder/builder.go:490 +0x118
github.com/strangelove-ventures/heighliner/cmd.queueAndBuild({{0x7ffc26e83678, 0x27}, 0x0, 0x1, {0x7ffc26e83660, 0x14}, {0xf415e8, 0x17}, 0x0, 0x0}, ...)
        /home/strangelove/heighliner/cmd/queue.go:119 +0xdca
github.com/strangelove-ventures/heighliner/cmd.BuildCmd.func1(0xc0001d5b80?, {0xf2dcac?, 0x9?, 0x9?})
        /home/strangelove/heighliner/cmd/build.go:130 +0x298
github.com/spf13/cobra.(*Command).execute(0xc0001d5b80, {0xc0001462d0, 0x9, 0x9})
        /home/strangelove/go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0xc0001d5900)
        /home/strangelove/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
        /home/strangelove/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/strangelove-ventures/heighliner/cmd.Execute({0x16fa440?, 0xda41c0?, 0xc000002340?})
        /home/strangelove/heighliner/cmd/root.go:32 +0xe5
main.main()
        /home/strangelove/heighliner/main.go:13 +0x2e

supplychainsecurity: apply supply chain security recommendations

Coming here from the results of a supply chain security analysis of this repository that we Orijtech Inc engaged Chainguard Inc, to perform on behalf of the Cosmos ecosystem. The report is at https://cyber.orijtech.com/scsec/cosmos-v1 or in PDF standalone https://cyber.orijtech.com/chainguard_cosmos_v1.pdf#page20

Tasks

Enable branch protection so that no one can just push directly to the main branch
Add dependabot and renovatebot
Enable GitHub Advanced Security
Add CodeQL and cosmos/gosec
Require multi-factor authentication for every contributor to the organization and repository per https://docs.github.com/en/organizations/keeping-your-organization-secure/managing-two-factor-authentication-for-your-organization/requiring-two-factor-authentication-in-your-organization
Need for reproducible builds. Please use apko https://github.com/chainguard-dev/apko https://www.chainguard.dev/unchained/introducing-apko-bringing-distroless-nirvana-to-alpine-linux

add go relayer to builds

Heighliner hits Github rate limits

Oct 14 21:01:16 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:16 Error queueing docker image builds for chain tendermint: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease
Oct 14 21:01:16 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:16 Error queueing docker image builds for chain terra: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease
Oct 14 21:01:17 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:17 Error queueing docker image builds for chain thorchain: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease
Oct 14 21:01:17 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:17 Error queueing docker image builds for chain umee: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease
Oct 14 21:01:17 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:17 Error queueing docker image builds for chain vidulum: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease
Oct 14 21:01:17 github-actions-heighliner buildLoop.sh[3535277]: 2022/10/14 21:01:17 Error queueing docker image builds for chain wasm: error parsing github releases response: {"message":"API rate limit exceeded for 104.197.32.130. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}, error: json: cannot unmarshal object into Go value of type []cmd.GithubRelease

Feat: Accept empty build targets

Re-open/update this PR: #7

dig

Chain.yaml:

# Dig
- name: dig
  github-organization: notional-labs
  github-repo: dig
  language: go
  build-target: install
  binaries:
    - /go/bin/digd

Error:

...
go: downloading github.com/zondax/hid v0.9.0
# github.com/zondax/hid
In file included from /go/pkg/mod/github.com/zondax/[email protected]/hid_enabled.go:22:
/usr/include/sys/poll.h:1:2: warning: #warning redirecting incorrect #include <sys/poll.h> to <poll.h> [-Wcpp]
    1 | #warning redirecting incorrect #include <sys/poll.h> to <poll.h>
      |  ^~~~~~~
# github.com/CosmWasm/wasmvm/api
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: librt.so.1, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libpthread.so.0, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libm.so.6, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libdl.so.2, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libc.so.6, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: warning: ld-linux-x86-64.so.2, needed by /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so: undefined reference to `memset@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/!cosm!wasm/[email protected]/api/libwasmvm.so: undefined reference to `pthread_cond_signal@GLIBC_2.3.2'

CMD:
./heighliner build -c dig -v v3.0.0

NOTES:

Adding:

  build-env:
    - BUILD_TAGS=muslc

produces same error.

strangelove-ventures / heighliner Goto Github PK

heighliner's People

Contributors

Stargazers

Watchers

Forkers

heighliner's Issues

GH Action logs

Examples

With built-in chain config

Custom chain config

Steps to reproduce:

Actual behavior:

Expected behavior:

Relevant Trace:

Title

Description

Proposed Feature Details

Example Output

Tasks

Recommend Projects

Recommend Topics

Recommend Org