Hello, any plans to support flutter cloud builder ? I would to use cloudbuild.yaml in my CI to build new release of my flutter application :)

It would be really nice to not have to build these images, but just reference using something like:
gcr.io/cloud-builders-community/

Understood that they come with no warranty....

Affected builder image

gcr.io/cloud-builders-community/docker-compose

Expected Behavior

Current directory volumes mounted via docker-compose.yml to not be empty.

Actual Behavior

Mounted volume is empty.

Steps to Reproduce the Problem

cloudbuild.yaml

steps:

- name: 'docker.io/tomatosource/docker-compose'
  id: test
  args:
    - 'up'
    - '--force-recreate'
    - '--abort-on-container-exit'
    - '--exit-code-from'
    - 'app'
  timeout: '300s'

docker-compose.yml

version: '3.5'

services:
  app:
    image: golang:alpine
    volumes:
      - .:/foo
    working_dir: /foo
    command: ls

Additional Info

Running locally via cloud-build-local --config=cloudbuild.yaml --dryrun=false .

Having some issues with the copy-licenses step which later on gives me this issue

Step #1: Step 16/18 : COPY licenses licenses
Step #1: COPY failed: stat /var/lib/docker/tmp/docker-builder155253068/licenses: no such file or directory

I've tried both using the env var and the gs bucket. Am I missing something? Thank you.

I am trying to use fabric-beta-dist-cloudbuild.yaml in android cloud builder for beta distribution but I can't figured out the value of the variables in your yaml filel like

• ${_SIGNING_BUCKET_PATH}
• ${_FABRIC_API_SECRET_BUCKET_PATH}
• ${_EMAIL_DISTRIBUTION_GROUP}
• _BUILD_TYPE_CAPITALIZED

Greetings!

I'm having trouble connecting to a container after it has been started via docker-compose (following the GCB docs). I've also found some code to go with those steps from the docs here.

I'm likely just missing what the hostname becomes off of the docker-compose up step (and it's not as simple as the container_name).

Minimally reproducible compose is:

version: '3.4'
services:
  web-server:
    image: nginx
    container_name: web-server
    ports:
    - "8080:80"
networks:
  default:
    external:
      name: cloudbuild

and cloudbuild is

steps:
- name: 'docker/compose:1.22.0'
  args: ['up', '-d']
- name: 'gcr.io/cloud-builders/docker'
  args: ['run', 'byrnedo/alpine-curl', 'web-server:8080']

Thanks!

Hi,

I was creating a custom build step based on the firebase builder and also testing Container Analysis API and it detected several vulnerabilities in this image which were caused by the official npm builder

Once I changed the base image in the custom build step's Dockerfile to use latest Node LTS (Boron) the vulnerabilities were gone.

FROM gcr.io/cloud-builders/npm
-->
FROM node:boron

BR
Masi

Git branches uses names like feature/new-feature, the slash is incompatible with docker image tags so i want to use a regex (or other means) to replace the slash in the $BRANCH_NAME with an underscore. This is possible in bash using the following:

BRANCH_NAME=feature/test echo ${BRANCH_NAME/\//_}

is there any way to create an env variable with the branch name escaped in cloudbuild.yaml?

Affected builder image

remote-builder

Expected Behavior

The main build uses scp to copy files to the remote machine.

Actual Behavior

ssh: connect to host 35.238.52.245 port 22: Connection timed out

Additional Info

It's probably just a problem with the setup of the GCP project?

Add a builder for compodoc. Compodoc is a tool used for generating documentation for your angular projects.

At the bottom of the android/examples/build-test-deploy.yaml file you can find the following comment and configuration:

# This build requires more than 3.75 GB of memory, so I have to use a HIGHCPU machine
# which has 30 GB of memory.  This means I can give Gradle lots of processes to run
# highly parallelized.

[...]

options:
  machineType: 'N1_HIGHCPU_8'

According to the Google cloud machine types documentation, the n1-highcpu-8 only has 7.2gb of RAM and NOT 30gb.

Am I missing something? Is GCB using the same machine types than GCC?

Hi there : )

When triggering the secondary build to monitor the state of the main build, the timeout of the main build is ignored.

So even if in my cloudbuild.yaml I set up a build time out of 900s, if my build happens to go over 600s no notification will be sent because the monitoring build will time out after 600s.

I've quickly hacked this in trigger.go hardcoding my own time out when creating the trigger:

		Tags: []string{"slackbot"},
		Timeout: "900s", // <--- I added this

Ideally Slackbot should pull the timeout information from the Cloud Build API in order to use the same or maybe slightly higher. But I haven't checked that this is possible and my Go knowledge is non-existent!

Thanks,

Juan

Apologies if using an issue to ask this is the wrong place.

Currently bq is a community builder i.e. located here.

Would it be worth considering bq as a pre-built builder? This is the case for the similar gsutil builder, and it feels to me like bq should have the same privileges too.

BigQuery is a native GCP tool, and it's the core tool to use on GCP for data and analytics. So having it as a pre-built image might make more sense.

The instructions on the firebase image state that the user should use the firebase login:ci command locally to generate a token, encrypt the token with KMS, then store that encrypted token in the cloudbuild.yaml file. These steps are unnecessary and incur additional costs because of the use of KMS to encrypt the secrets.

You can instead directly grant the Build service account user (it'll have the name <project-number>@cloudbuild.gserviceaccount.com) the following roles and it will be able to perform firebase deploys, at least of the 'Firebase Hosting' product. Some additional roles might need adding for other Firebase products.

1. Cloud Build Service Account
2. Firebase Admin
3. API Keys Admin

You also need to enable these APIs for the project if they aren't already enabled:

1. Cloud Resource Manager API
2. Firebase Management API
3. Firebase Hosting API

Is there a reason the firebase login:ci approach is promoted here? Is it preferred for some reason, or just outdated?

Affected builder image

gcr.io/cloud-builders-community/remote-builder

Expected Behavior

It can accept an INSTANCE_ARGUMENT with a space such as "--min-cpu-platform=Intel Haswell"

Actual Behavior

The Space in the argument causes the create command to break.

Steps to Reproduce the Problem

1.Supply a --min-cpu-platform=Intel Haswell to the remote builder as an INSTANCE_ARGUMENT.
2.Supply a --min-cpu-platform='Intel Haswell' to the remote builder as an INSTANCE_ARGUMENT.
3.Supply a --min-cpu-platform="Intel Haswell" to the remote builder as an INSTANCE_ARGUMENT.

Each fails with something like:

- Invalid value for field 'resource.minCpuPlatform': '"Intel'. Invalid CPU platform.
- Invalid value for field 'resource.name': 'Haswell"'. Must be a match of regex '(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)'

Additional Info

In the Android simple-cloudbuild.yaml, the example writes the the output to a fixed path based on project, branch, and git commit. If a build is run more than once on the same commit, the build outputs will be overwritten.

By adding the $BUILD_ID to the output path, the artifacts from previous invocations of the same build would be preserved.

When following the helm/README.md, I'm getting the following error when using could build during building:
standard_init_linux.go:195: exec user process caused "no such file or directory"

This is due to the produced docker image not being correct.
After investigation, this appears to be a line ending issue.

When converting line endings of all files from a windows line-ending (\r\n) to unix line ending (\n) before calling the gcloud builds submit command, a working image is obtained.

I'm not sure what a correct approach would be to correct this problem, since line endings seem to be dependent on the local git config.

At the very least, this could be mentioned in the helm/README.md

I haven't checked, but the same issue could be present in other community builders.

Affected builder image

gcr.io/cloud-builders-community/kaniko

Expected Behavior

When building images with kaniko in cloudbuild.yaml the images should be found at the end of the build.

cloudbuild.yaml

steps:
  # Build image and push to gcr with kaniko
  - name: gcr.io/kaniko-project/executor:latest
    args: [
      "--context=/workspace/api",
      "--destination=gcr.io/<$PROJECT_ID>/api:latest",
      "--destination=gcr.io/<$PROJECT_ID>/api:$SHORT_SHA",
      "--cache=true",
      ]

# List built images in cloudbuild history
images:
  - "gcr.io/<$PROJECT_ID>/api:$SHORT_SHA"
  - "gcr.io/<$PROJECT_ID>/api:latest"

tags: ["$BRANCH_NAME"]

Actual Behavior

The images have been pushed to Container Registry and can be deployed via kubectl.

The cloudbuild fails with error that images cannot be found:

ERROR: failed to find one or more images after execution of build steps: 
["gcr.io/<$PROJECT_ID>/api:a4a36cd", "gcr.io/<$PROJECT_ID>/api:latest"]

Steps to Reproduce the Problem

1. Use the kaniko-project/executor builder in cloudbuild.yaml
2. Add images section to cloudbuild.yaml that verifies / pushes the images
3. Trigger build

Additional Info

We're currently using docker to build our containers and wanted to give kaniko a shot, it compiles much faster 🎉 but it seems there is a mismatch how the images section is handled when images are not being built by docker.

When submitting builds, following error is displayed.

ERROR: (gcloud.builds.submit) INVALID_ARGUMENT: invalid build: invalid .secrets field: secretEnv "FIREBASE_TOKEN" is used without being defined

secretEnv: ['FIREBASE_TOKEN'] is not required in cloudbuild.yaml
I will make PR.

When trying to use this image, using both gcr.io/cloud-builders/kustomize as well as gcr.io//gcloud (which is stated to use in the docs) results in no available images for Kustomize. Have the images been pulled?

Affected builder image

(e.g., gcr.io/cloud-builders-community/helm)

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Additional Info

Neither an example directory.

Affected builder image

gcr.io/cloud-builders-community/container-diff

Expected Behavior

gcloud builds submit . --config=cloudbuild.yaml creates the container-diff container

Actual Behavior

Nested errors

Steps to Reproduce the Problem

1. git clone https://github.com/GoogleCloudPlatform/cloud-builders-community.git
2. cd cloud-builders-community/container-diff
3. gcloud builds submit . --config=cloudbuild.yaml

Additional Info

It appears that the Bazel configuration is simply out of date as some of the syntax and targets don't seem to be current.

When running cloudbuild.yaml in the root folder, the android builder does not pass because the required environment configuration is not provided.

I can't see the output from this project's build, so it's possible this conclusion is wrong; but that seems to be what's going on.

When trying to use remote-builder I noticed that after Thursday(01/31/2019) I started getting this error in my builds.

The builder would create the VM and then try to connect with ssh and would get a warning that WARNING: Using OS Login user [<user_name>] instead of default user [${USERNAME}]. From what I can tell the first username is the username for the google build account and the second one is the one that I tried to get it to use with the USERNAME environment variable. This issue seems to start on 02/01/2019 according to my cloud build logs.

Add Apache JMeter builder for executing JMX scripts

If someone's interested in a Scala builder for Cloud Build, let's get in touch.

There are multiple problems with the current one, the biggest maybe that it fails to load in the libraries that make sbt and Scala (it's kind of the beef, right?).

I had a go at my own approach, but it needs some testing. You can find it here: https://github.com/akauppi/sbt-scala-builder

Thanks for the help.

The Helm builder uses version 2.10. It would be nice to update it to the recent version 2.11

https://github.com/helm/helm/releases/tag/v2.11.0

Moved from GoogleCloudPlatform/cloud-builders#275

I will start work on this in the next couple of days.

The root cloudbuild submits builds for every sub-build in the project at once. Cloudbuilder guidelines suggest ~10 concurrent builds per project is the limit.

See #72 for more detail and a proposed solution.

Hi,

I am exploring using cloudbuild for building out terraform infra but was wanting to ask for help what appears to be a simple request.

My .tf files are in sub directories within the checkout source code from github - in order to do a terraform init, I need to 'cd' (docker WORKDIR) to a sub directory before executing a specific terraform command.

Is there a basic cloudbuild 'step' for this Im missing to do this before executing an actual terraform command? Im not a docker expert (far from it!) so any help for a newbie appreciated.

Thanks!

If docker-compose build step is used to create and detach a composed YAML how do I access the containers in next step. E.g. I create a "webserver" container and expose port 80 of the container -> localhost. However in the next "ubuntu" build step I cannot access this container e.g. both localhost:8000 and webserver:80 don't work.

The NPM integration test example makes it seem as if the containers created from previous compose step should be accessible next steps. But since each step runs in a "container" do I need to explicitly forward the host/port ?

running gcr.io/$PROJECT_ID/docker-compose in cloud build step fails

ERROR: build step 2 "docker/compose" failed: error pulling build step 2 "docker/compose": exit status 1
ERROR
ERROR: failed to pull because we ran out of retries.```

The current firebase builder example doesn't work because of the underscore in FIREBASE_TOKEN, due to GoogleCloudPlatform/cloud-builders#413

It currently fails with ERROR: (gcloud.builds.submit) INVALID_ARGUMENT: invalid build: invalid .secrets field: secretEnv "FIREBASE_TOKEN" is used without being defined

A work-around is to use sentence case for the part of the variable after the underscore. Here's what the example would look like:

steps:
- name: 'gcr.io/valr-ci/firebase'
  entrypoint: /bin/bash
  args: ['-c', 'firebase list --token $$FIREBASEToken']
  secretEnv: ['FIREBASEToken']
secrets:
- kmsKeyName: 'projects/[PROJECT_ID]/locations/global/keyRings/cloudbuilder/cryptoKeys/firebase-token'
  secretEnv:
    FIREBASE_TOKEN: '<ENCRYPTED_TOKEN>'

$ gcloud --version
Google Cloud SDK 224.0.0
bq 2.0.36
core 2018.11.02
gsutil 4.34

cc @petomalina @imjasonh

Affected builder image

gcr.io/cloud-builders-community/sonarqube

Expected Behavior

successful build

Actual Behavior

Step #2: START: Running sonar-scanner-cli on Tue Mar 19 20:26:02 UTC 2019
Step #2: INFO: Scanner configuration file: /opt/sonar-scanner-3.2.0.1227-linux/conf/sonar-scanner.properties
Step #2: INFO: Project root configuration file: NONE
Step #2: INFO: SonarQube Scanner 3.2.0.1227
Step #2: INFO: Java 1.8.0_201 Oracle Corporation (64-bit)
Step #2: INFO: Linux 4.15.0-1028-gcp amd64
Step #2: INFO: User cache: /root/.sonar/cache
Step #2: INFO: ------------------------------------------------------------------------
Step #2: INFO: EXECUTION FAILURE
Step #2: INFO: ------------------------------------------------------------------------
Step #2: INFO: Total time: 1.148s
Step #2: INFO: Final Memory: 4M/56M
Step #2: INFO: ------------------------------------------------------------------------
Step #2: ERROR: Error during SonarQube Scanner execution
Step #2: ERROR: null
Step #2: ERROR: Caused by: Could not initialize NSS
Step #2: ERROR: Caused by: /usr/lib/libnss3.so
Step #2: ERROR: 
Step #2: ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

Steps to Reproduce the Problem

1. build image from source
2. add build step to cloudbuilder
3. run with the following options

  - name: "gcr.io/my-project/sonar-scanner:latest"
    args:
      - '-Dsonar.projectKey={projectKey}'
      - '-Dsonar.organization={organization}'
      - '-Dsonar.sources=src'
      - '-Dsonar.host.url=https://sonarcloud.io'
      - '-Dsonar.login={token}'
      - '-Dsonar.exclusions=node_modules/**'
      - '-Dsonar.typescript.lcov.reportPaths=coverage/lcov.info'
      - '-Dsonar.coverage.exclusions=src/**/*.spec.ts'
    dir: "functions"

Additional Info

This works if I run from my local computer, just fails in cloudbuilder

I'm trying to run the firebase cloud-builder however I get this error when it tries to run the step.

Step #1: docker: Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: "/firebase.bash": permission denied": unknown.

Any ideas or workarounds?

thanks,
--mike

https://github.com/github/hub

Authenticate with GitHub access tokens in secretEnv vars

Add a builder for performing Sonarqube analysis

Affected builder image

gcr.io/cloud-builders-community/helm

Expected Behavior

• Deploy complete!

Actual Behavior

Pulling image: gcr.io/$PROJECT_ID/firebase
Using default tag: latest
latest: Pulling from $PROJECT_ID/firebase
e79bb959ec00: Already exists
d4b7902036fe: Already exists
1b2a72d4e030: Already exists
d54db43011fd: Already exists
69d473365bb3: Already exists
6e2490ee2dc8: Already exists
a379571b1bfc: Already exists
7b9a41966503: Already exists
f4b1c42f5c2b: Pulling fs layer
17b0c2359212: Pulling fs layer
715c6ee9925e: Pulling fs layer
17b0c2359212: Verifying Checksum
17b0c2359212: Download complete
715c6ee9925e: Verifying Checksum
715c6ee9925e: Download complete
f4b1c42f5c2b: Verifying Checksum
f4b1c42f5c2b: Download complete
f4b1c42f5c2b: Pull complete
17b0c2359212: Pull complete
715c6ee9925e: Pull complete
Digest: sha256:6445a29e9a3a5095e0ce7f01246d99a64d7942c911ef350ea7a7ee4906319617
Status: Downloaded newer image for gcr.io/$PROJECT_ID/firebase:latest
standard_init_linux.go:190: exec user process caused "no such file or directory"

Steps to Reproduce the Problem

1. Build Docker image as documented
2. Submit build via Google Cloud Build
3. Build fails at final step as above

Additional Info

I have replaced the name of my actual project above with $PROJECT_ID.
Running firebase deploy from my laptop works.
Cloudbuild.yaml below
steps:

Install

• name: 'gcr.io/cloud-builders/npm'
  args: ['install']

Build

• name: 'gcr.io/cloud-builders/npm'
  args: ['run', 'build', '--prod']

Deploy

• name: 'gcr.io/$PROJECT_ID/firebase'
  args: ['deploy']

As I understand cloud-build use "workspace" directory as a volume, but when I download files and save it into this folder them doesn't exist on next steps. Could you please take a look at this?

In the Android gradle-build script, the script is using gradle when the best practice is to use the gradle wrapper gradlew.

The version of gradle being installed by the Dockerfile is version 4.4 which won't work with the latest version of the Android Gradle plugin. Using gradlew avoids the need to keep updating the Dockerfile.

Caching should still be fine, as the gradle download ends up in the directory that is already being cached.

the kustomize builder is great but it assumes a GKE environment.

This means that you can use it to simply render manifests.

To add Cloud Builder for cft (Cloud Foundation Toolkit).
https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/community/cloud-foundation/docs/userguide.md

Affected builder image

(e.g., gcr.io/cloud-builders-community/helm)

Expected Behavior

The image pulls down and is available for use on Cloud Build

Actual Behavior

When trying to use this image, using both gcr.io/cloud-builders/kustomize as well as gcr.io//kustomize (which is stated to use in the docs) results in no available images for Kustomize.

Steps to Reproduce the Problem

1. Use either gcr.io/cloud-builders/kustomize
2. or gcr.io//kustomize as a image for cloudbuild to utilize

Additional Info

Were the images pulled? There's none available?

Configuring the hugo builder as per the readme...

steps:
- name: 'gcr.io/$PROJECT_ID/hugo'

gives the following error on google cloud build:

Pulling image: gcr.io/<project-id>/hugo
Using default tag: latest
Error response from daemon: manifest for gcr.io/<project-id>/hugo:latest not found

where is my project Id.

Hi,

Been looking for resources regarding limiting concurrent build execution per branch/tag/repo at a time but found none. Is this currently supported or is there any workaround to achieve this?

Thank you!

I was following the guide for building an android image with sdk version 28.
apply the following command

gcloud builds submit --config=cloudbuild.yaml --substitutions=_ANDROID_VERSION=28
yelided the following errors

Step #0: Accept? (y/N): Skipping following packages as the license is not accepted:
Step #0: SDK Patch Applier v4
Step #0: Android SDK Platform-Tools
Step #0: Android SDK Build-Tools 28.0.3
Step #0: The following packages can not be installed since their licenses or those of the packages they depend on were not accepted:
Step #0:   build-tools;28.0.3
Step #0:   patcher;v4
Step #0:   platform-tools
Step #0:   extras;google;m2repository
Step #0:   extras;google;google_play_services
ERROR: (gcloud.builds.submit) build ${BUILD_ID} completed with status "FAILURE"

after investaging the issue was in the Dockerfile-base in runing the following step

RUN sdkmanager \
    "tools" \
    "platform-tools" \
    "extras;android;m2repository" \
    "extras;google;m2repository" \
    "extras;google;google_play_services" \
    "build-tools;28.0.3"

changing the step to

RUN yes | sdkmanager \
    "tools" \
    "platform-tools" \
    "extras;android;m2repository" \
    "extras;google;m2repository" \
    "extras;google;google_play_services" \
    "build-tools;28.0.3"

fixed the issue

https://github.com/alecthomas/gometalinter