circleci-public / aws-ecr-orb Goto Github PK
View Code? Open in Web Editor NEWCircleCI orb for interacting with Amazon's Elastic Container Registry (ECR)
Home Page: https://circleci.com/orbs/registry/orb/circleci/aws-ecr
License: MIT License
CircleCI orb for interacting with Amazon's Elastic Container Registry (ECR)
Home Page: https://circleci.com/orbs/registry/orb/circleci/aws-ecr
License: MIT License
6.3.0
my setup involves the IAM ci
user used by circle having no permissions except assumeRole, and a ci
role having permissions to ECR, as common practice for segregating access control.
I had assumed that the profile-name
param in concert with a custom awscli config
file (containing a profile definition that specifies a role_arn
) would serve exactly this purpose but I've been unable to find a way to make it work.
seeing issue #9 open I'd think that this use case is not supported at all, is this the case?
I believe this user-role segregation with assumeRole is a common and recommended security pattern and it would be great to see it supported in an official orb.
6.0.0
I'm trying to call the build-and-push-image
job, but was wondering whether I can pass in a target image for the build step?
Essentially I’m using a multi-stage build in my Dockerfile, where the last image contains all of the test code. I would like to push the previous image to ECR, but can’t see how to do this using the build-and-push-image
job.
commands:
build_and_push_image_to_ecr:
description: "Build Docker image and push to ECR"
steps:
- aws-ecr/build-and-push-image:
account-url: AWS_ECR_ACCOUNT_URL
create-repo: true
repo: $SERVICE_REPO
tag: $CIRCLE_SHA1
Would it be possible to specify a target image?
If there was an option to use docker build's --cache-from here:
https://github.com/CircleCI-Public/aws-ecr-orb/blob/master/src/orb.yml#L73
It would presumably cut build time significantly in the same way Circle's proprietary layer caching does, but more reliably
Reference:
https://semaphoreci.com/docs/docker/docker-layer-caching.html
https://docs.docker.com/edge/engine/reference/commandline/build/
Would be great to have this Orb support https://circleci.com/docs/2.0/docker-layer-caching/
3.1.0
jobs:
java-build:
executor: java-with-postgres-executor
steps:
- checkout
...
- persist_to_workspace:
root: /tmp/java-build
paths:
- docker/web/*.war
workflows:
version: 2
build-test-and-deploy:
jobs:
...
- aws-ecr/build_and_push_image:
checkout: true
attach-workspace: true
workspace-root: /tmp/java-build
account-url: AWS_ECR_ACCOUNT_URL
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
dockerfile: /tmp/java-build/docker/web/Dockerfile
path: /tmp/java-build/docker/web
region: AWS_DEFAULT_REGION
repo: "ecs-circleci-qa-repository"
tag: "${CIRCLE_SHA1}"
I got
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/java-build/docker/web/Dockerfile: no such file or directory
Exited with code 1
https://circleci.com/gh/gitenter/gitenter/388
but change to .
the job succeed.
- aws-ecr/build_and_push_image:
checkout: true
attach-workspace: true
workspace-root: .
account-url: AWS_ECR_ACCOUNT_URL
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
dockerfile: ./docker/web/Dockerfile
path: ./docker/web
region: AWS_DEFAULT_REGION
repo: "ecs-circleci-qa-repository"
tag: "${CIRCLE_SHA1}"
https://circleci.com/gh/gitenter/gitenter/393
If it is equivalent to
working_directory: /tmp/java-build
steps:
- checkout
- attach_workspace:
at: /tmp/java-build
workspace-root
should support arbitrary path rather than just .
.
The whole repo has only one release v6.1.0-minor
currently
How to manage the older versions? How to trace the changes between different versions?
got different issues with different orb versions
With sample (https://circleci.com/blog/orbs-aws-ecr/) by using 0.0.4
, this pipeline work with command circleci config process .circleci/config.yml
I'd like to manage the region, account-url, repo and tag name with CONTEXT variables or CIRCLE variables.
workflows:
build_test_deploy:
jobs:
- aws-ecr/build_and_push_image:
region: ${AWS_DEFAULT_REGION}
account-url: ${AWS_ACCOUNT_ID}.dkr-ecr.${AWS_DEFAULT_REGION}.amazonaws.com
repo: ${REPO_NAME}
tag: 1.0.${CIRCLE_BUILD_NUM}
between version >0.0.4 and < 5.0.0, I got below error
$ circleci config process .circleci/config.yml
Error: Error calling workflow: 'build_test_deploy'
Error calling job: 'aws-ecr/build_and_push_image'
Type error for argument region: expected type: env_var_name, actual value: "${AWS_DEFAULT_REGION}" (type string)
Type error for argument account-url: expected type: env_var_name, actual value: "${AWS_ACCOUNT_ID}.dkr-ecr.${AWS_DEFAULT_REGION}.amazonaws.com" (type string)
for version >5.0.0 to latest version 6.1.0, I got below error
$ circleci config process .circleci/config.yml
Error: Error calling workflow: 'build_test_deploy'
Cannot find a definition for job named aws-ecr/build_and_push_image
should work as 0.0.4
... ignore. sorry. somehow I copied the wrong orb version
6.7.0
I am changing my app to use a base build image from a aws ecr private repo and then after running everything pushes the app image to another ecr.
I read the documentation in using private aws ecr here and everything worked on the apps that don't use the orb to push to another instance. The only one that uses the orb it fails as I show below.
My .circleci/config.yml file:
version: 2.1
orbs:
aws-ecr: circleci/[email protected]
jobs:
test:
working_directory: ~/circle
docker:
- image: $AWS_BUILD_IMAGE_ECR_ACCOUNT_URL
aws_auth:
aws_access_key_id: $AWS_BUILD_IMAGE_ACCESS_KEY_ID
aws_secret_access_key: $AWS_BUILD_IMAGE_SECRET_ACCESS_KEY
# .... run tests, deployment stuff, etc
workflows:
version: 2
release:
jobs:
- test
- aws-ecr/build-and-push-image:
name: push_app_image
dockerfile: Dockerfile
account-url: AWS_ECR_ACCOUNT_URL
region: AWS_DEFAULT_REGION
repo: "name-of-the-repo"
tag: "APP_${CIRCLE_SHA1}"
This is the error that raises in the beginning of the CI:
error authentication with ECR: UnrecognizedClientException: The security token included in the request is invalid.
status code: 400
I already verified a lot of times the env values of AWS_BUILD_IMAGE_ECR_ACCOUNT_URL and
AWS_ECR_ACCOUNT_URL and the values are okay as I can login using the aws
cli for each different ecr.
The reason that I am posting here is that all the other repositories which I use the build image passes in the CircleCI and only this particular app that use this orb does not pass so I wonder how can I achieve what I want to achieve.
Is this a bug? Sorry if this is not the right place to post this issue.
I know the orbs looks for default env vars like AWS_ECR_ACCOUNT_URL
so I wonder if CircleCI is looking to AWS_ECR_ACCOUNT_URL
as well for the base image instead of looking for AWS_BUILD_IMAGE_ECR_ACCOUNT_URL
(the right one that I specified).
The next step that I will try is to change "AWS_ECR_ACCOUNT_URL" to something else but I feel that I should raise this issue in case someone already had or will have this issue too.
Any thoughts?
When updating from the example, 0.0.2 to 6.0.0. Circle is throwing an error:
Cannot find a definition for job named aws-ecr/build_and_push_image
Job should run.
my config:
version: 2.1
orbs:
aws-ecr: circleci/[email protected]
workflows:
build-and-deploy:
jobs:
- aws-ecr/build_and_push_image:
account-url: "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
repo: "${AWS_REPO}"
region: ${AWS_DEFAULT_REGION}
tag: "${CIRCLE_SHA1}"
3.1.0
If I use aws-ecr/build_and_push_image
in workflows
as a "job", then everything goes fine.
workflows:
version: 2
build-test-and-deploy:
jobs:
...
- aws-ecr/build_and_push_image:
...
However, if I use it as a "command" in my customized job, I'll get error messages.
jobs:
- qa-deploy:
docker:
- image: circleci/postgres:11
environment:
POSTGRES_PASSWORD: postgres
steps:
- checkout
- aws-ecr/build_and_push_image:
...
workflows:
version: 2
build-test-and-deploy:
jobs:
...
- qa-deploy
#!/bin/sh -eo pipefail
# Error calling workflow: 'build-test-and-deploy'
# Error calling job: 'qa-deploy'
# Cannot find a definition for command named aws-ecr/build_and_push_image
#
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
I think it will be nice if aws-ecr/build_and_push_image
can be used as a command. By doing so we can hide the detailed steps in my customized job, rather than showing a bloated list of steps in my workflow.
Notice that it is supported by circleci/aws-s3
https://circleci.com/orbs/registry/orb/circleci/aws-s3
version: 2.1
orbs:
aws-s3: circleci/[email protected]
jobs:
build:
docker:
- image: 'circleci/python:2.7'
steps:
- checkout
- run: mkdir bucket && echo "lorum ipsum" > bucket/build_asset.txt
- aws-s3/sync:
from: bucket
to: 's3://my-s3-bucket-name/prefix'
arguments: |
--acl public-read \
--cache-control "max-age=86400"
overwrite: true
- aws-s3/copy:
from: bucket/build_asset.txt
to: 's3://my-s3-bucket-name'
arguments: '--dryrun'
4.0.1
Expected repo
to be a URI as per the documentation. Instead, it's the repo name.
I left account-url
blank, but the interpolated URI becomes << parameters.account-url >>/<< parameters.repo >>:<<
Since repo
is the only required parameter, I would expect it to be a URI and not a repository name.
6.1.0
description for account-url
has a typo:
It is:
{awsAccountNum}.dkr-ecr.us-west-2.amazonaws.com
It should be:
{awsAccountNum}.dkr.ecr.us-west-2.amazonaws.com
Applies to >=4.0.2
Would be neat to have parameters for AWS account IDs to give secondary access to created repos. If a config has create-repo: true
, there could be another property secondary-account-ids
that takes a list of AWS account IDs to add to a policy that allows for image pulling from those accounts.
By using the orb and listing create-repo: true
and secondary-account-ids
, my build should build and push an image to a newly-created repo, and my secondary accounts should be able to pull from the newly-created repo.
3.0.0
Docker build fails with multi-stage copy
---> ff87c2f09e67
Removing intermediate container c0803b11e91c
Step 24/28 : COPY --from=build-prod /usr/src/app/node_modules /usr/src/app/node_modules/
COPY failed: stat /var/lib/docker/overlay2/7afb654f56fa9ec29e75fc7b8b8d24285408646a63bd1a34a6ce395c848447e8/merged/usr/src/app/node_modules: no such file or directory
Exited with code 1
I may not be understanding this properly... perhaps there is something to do with the executor used with this build to not have a docker engine that supports multi-stage build commands / functionality?
Here is my config.yml
orbs:
aws-ecr: circleci/[email protected]
version: 2.1
workflows:
build_and_push_image:
jobs:
- aws-ecr/build_and_push_image:
repo: ${AWS_ECR_REPO_NAME}
tag: testing-alpha.0
Update
I've added the recommended following /
to the path names for copying folders, same issue.
6.2.0
My .circle/config.yml
file:
orbs:
aws-ecr: circleci/[email protected]
version: 2.1
workflows:
build_and_push_image:
jobs:
- aws-ecr/build-and-push-image:
account-url: AWS_ACCOUNT
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
create-repo: true
region: AWS_REGION
repo: myRepo
Build should get past this step.
It will be really nice if we can test our docker image (through goss test) after build the image.
Refer to:
6.7.0
build-and-deploy is failing on "build docker image":
#!/bin/bash -eo pipefail
docker_tag_args=""
IFS="," read -ra DOCKER_TAGS <<< "latest"
for tag in "${DOCKER_TAGS[@]}"; do
docker_tag_args="$docker_tag_args -t $AWS_ECR_ACCOUNT_URL/temi-web-platform:$tag"
done
docker build \
\
-f ./Dockerfile \
$docker_tag_args \
.
`
`invalid argument "/*****************:latest" for t: invalid reference format
See 'docker build --help'.
Config:
aws-ecr/build-and-push-image:
executor: default_exec
account-url: AWS_ECR_ACCOUNT_URL
region: AWS_REGION
repo: temi-web-platform
tag: latest
requires:
- test
context: temi-web-platform-global
filters:
branches:
only:
- integration
- dev-aviram
Link to failing build: https://app.circleci.com/jobs/github/ROBOTEAM-HOME/web-platform/42
Docker build completes successfully.
If the profile-name
setting is overridden, the build fails.
Applies to <=4.0.0
When new versions are published, It would be nice to see what has been changed from one version to another. Particularly when it is a breaking change (change in major).
CHANGELOG.md exists and is up to date, possibly following this: https://keepachangelog.com/en/1.0.0/ as part of your release process, make sure that the file is updated.
6.0.6
I tried building and pushed an image to ECR which is in a subfolder in my repo by providing the path
parameter (path: './dir'
). That failed with:
#!/bin/bash -eo pipefail
docker_tag_args=""
IFS="," read -ra DOCKER_TAGS <<< "latest"
for tag in "${DOCKER_TAGS[@]}"; do
docker_tag_args="$docker_tag_args -t $AWS_ACCOUNT_URL/repo:$tag"
done
docker build \
\
-f Dockerfile \
$docker_tag_args \
./dir
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/circleci/project/Dockerfile: no such file or directory
Exited with code 1
CircleCI received exit code 1
It worked only when I added both dockerfile: './dir/Dockerfile
and path: './dir'
Ideally, just by setting the path parameter the Docker context including Dockerfile should be set. If not, it is not stated in the docs that the dockerfile parameter is required when setting the path.
- aws-ecr/build-and-push-image:
install_cli: false
I want to be able to skip this step , since it executed on every build and push, sometimes you have already done a aws-cli/setup or already pushed an image at step before and its just waste of time to instsall it again.
I know that there is a step to check if there is an aws already installed.
But when I use latest orb of aws-cli , it installs AWS CLI v2 and then ECR orb install V1 again...
3.0.0
When running a job with create-repo: true the job will fail on the second run because the repo already exists.
aws --region $AWS_REGION ecr create-repository --repository-name dummy-service --profile default
An error occurred (RepositoryAlreadyExistsException) when calling the CreateRepository operation: The repository with name 'dummy-service' already exists in the registry with id 'xxxxxxxxxxx'
Exited with code 255
According to the documentation the repo should be created if it does not exist, this would mean that the RepositoryAlreadyExistsException should be ignored.
1.0.0
I'm unable to use the aws-ecr/build_and_push_image command in a command or a job directly, it only works in the workflow.
Here's an example of my CircleCI config:
version: 2.1
orbs:
aws-ecr: circleci/[email protected]
commands:
build-push:
description: "Build Docker image and push to ECR"
parameters:
repo:
type: string
dockerfile:
type: string
steps:
- aws-ecr/build_and_push_image:
repo: << parameters.repo >>
dockerfile: << parameters.dockerfile >>
jobs:
build-push:
docker:
- image: circleci/python:3.7.1
steps:
- build-push:
repo: "demo"
dockerfile: "demo/Dockerfile"
workflows:
build-deploy:
jobs:
- build-push:
context: dev-global
When ran CircleCI errors out and outputs this:
#!/bin/sh -eo pipefail
# Error calling workflow: 'build-deploy'
# Error calling job: 'build-push-backend'
# Error calling command: 'build-push'
# Cannot find a definition for command named aws-ecr/build_and_push_image
#
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
The aws-ecr/build_and_push_image command should work in commands and jobs.
6.1.0
#!/bin/bash -eo pipefail
aws configure set aws_access_key_id
$AWS_ACCESS_KEY_ID
--profile default
Traceback (most recent call last):
File "/opt/circleci/.pyenv/versions/2.7.12/bin/aws", line 19, in
import awscli.clidriver
File "/opt/circleci/.pyenv/versions/2.7.12/lib/python2.7/site-packages/awscli/clidriver.py", line 36, in
from awscli.help import ProviderHelpCommand
File "/opt/circleci/.pyenv/versions/2.7.12/lib/python2.7/site-packages/awscli/help.py", line 20, in
from docutils.core import publish_string
File "/opt/circleci/.pyenv/versions/2.7.12/lib/python2.7/site-packages/docutils/core.py", line 246
print('\n::: Runtime settings:', file=self._stderr)
^
SyntaxError: invalid syntax
Exited with code 1
AWS would configure its credentials
aws-ecr: circleci/[email protected]
I am using aws-ecr obrs to build docker image, create an ECR repository if doesn't exist and push that docker image to ECR repository for .NET Framework legacy application. It is executing fine till the step of Build docker image. When it is trying to create an ECR repository, it is breaking at this step. It is unable to create ECR repository.
CircleCI Link:
https://app.circleci.com/pipelines/github/moodysanalytics/cmm-service/10/workflows/e6dbc71d-e9b0-43eb-9525-870ecacb35cd/jobs/9
After building the docker image, it should create an ECR repository and push the docker image to that ECR repository.
6.2.0
- aws-ecr/build-and-push-image:
repo: your-repo-name
tag: tagone,tagtwo
Results in:
docker build \
-f Dockerfile \
-t your-repo-name:tagone,tagtwo \
.
invalid argument your-repo-name:tagone,tagtwo" for t: invalid reference format
See 'docker build --help'.
Exited with code exit status 125
Docker build should succeed and the command produced should be:
docker build \
-f Dockerfile \
-t your-repo-name:tagone \
-t your-repo-name:tagtwo \
.
I guess it's the escaping done here https://github.com/CircleCI-Public/aws-ecr-orb/blob/master/src/commands/build-image.yml#L47
When I try locally it works just fine:
#!/bin/bash
docker_tag_args=""
IFS="," read -ra DOCKER_TAGS <<< "tagone,tagtwo"
for tag in "${DOCKER_TAGS[@]}"; do
docker_tag_args="$docker_tag_args -t image:$tag"
done
echo $docker_tag_args // -t image:tagone -t image:tagtwo
So the bash is valid and works. My conclussion would be that \<<<
is the problem. But I don't know what this is run through before it's run? It appears it's trying to get around parameter replacement for << ... >>
.
2.0.3
I'm trying to deploy a Haskell application but I'm failing to copy the executable file from one job to the other. This is my CircleCI config file:
version: 2.1
orbs:
haskell: haskell-works/[email protected]
aws-ecr: circleci/[email protected]
workflows:
build_and_publish_bill_statement_gen:
jobs:
- haskell/build:
name: GHC 8.6.3
executor: haskell/ghc-8_6_3
cabal-file: bill-statement-gen.cabal
- aws-ecr/build_and_push_image:
requires: [GHC 8.6.3]
context: staging-microservices
region: AWS_DEFAULT_REGION
repo: bill-statement-gen
attach-workspace: true
And my Dockerfile
:
FROM quay.io/haskell_works/ghc-8.6.3
ADD /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.3/bill-statement-gen-0.1.0.0/x/bill-statement-gen-exe/build/bill-statement-gen-exe/bill-statement-gen-exe /usr/bin/bill-statement-gen-exe
ADD /root/project/config/app.dhall /config/app.dhall
CMD bill-statement-gen-exe
I'm not sure on whether attach-workspace
is the option I'm looking for or not but I'd expect to have access to the executable file generated in the previous job haskell/build
.
I get this error:
Step 2/5 : ADD /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.3/bill-statement-gen-0.1.0.0/x/bill-statement-gen-exe/build/bill-statement-gen-exe/bill-statement-gen-exe /usr/bin/bill-statement-gen-exe
ADD failed: stat /var/lib/docker/tmp/docker-builder029745061/root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.3/bill-statement-gen-0.1.0.0/x/bill-statement-gen-exe/build/bill-statement-gen-exe/bill-statement-gen-exe: no such file or directory
Exited with code 1
Some Dockerfiles contain arguments:
ARG SOME_ARG="default_value"
and are filled in during building: docker build --build-arg SOME_ARG=my_value
Can we add support for a list of build arguments in this orb?
Add parameter to allow docker layer caching
- when:
condition: <<parameters.setup-remote-docker>>
steps:
- setup_remote_docker:
docker_layer_caching: <<parameters.docker-layer-caching>>
When using a custom docker executor, setup_remote_docker is most useful when you can enable docker_layer_caching to speed up builds. This parameter is false
by default, so we should have a way to enable it in the orb parameters. Otherwise it's impossible to enable when using this orb.
I am already running a docker image with AWS CLI v2 installed. Using this orb causes AWS CLI v1 to be installed. While the AWS CLI v2 does change the login procedure I feel like a switch statement could fix that.
I envision this change:
- aws-ecr/build-and-push-image:
...
install-aws-cli: false
aws-cli-version: v2
...
The default for install-aws-cli
could be set to true
which maintains backward compatibility. The default for aws-cli-version
could be set to v1
which would also maintain backward compatibility.
The re-installation feels redundant and also adds time (and thus uses credits) that isn't necessary. I have a project requirement to use AWS CLI v2 and to have it baked into my main image to use a specific version. That means it's not as simple as choosing to use a different base image so I can also use this orb.
This kind of increased flexibility would help folks out as they transition from AWS CLI v1 to v2 and also help reduce costs for folks on CircleCI.
3.1.0
I'm not being able to set the TAG parameter on runtime. On my workflow, I got a job that downloads the project and creates a release. I'd like to take the release number and pass to build_and_push_image as an environment variable. I've tried the following
-release_job
export TAG_VERSION=`git describe --abbrev=0 --tags`
then on the ecr job
- aws-ecr/build_and_push_image:
tag: TAG_VERSION
Also tried to set the environment variable using
echo 'export TAG_VERSION=
git describe --abbrev=0 --tags' >> $BASH_ENV
Also tried tag: ${TAG_VERSION}
There should be a way to pass args for the build being generated. Currenty cannot find such documentation for the same. If there's a way please point out.
We need to pass the sensitive data from env to args for the docker builds.
Type check error happens when version is above 0.0.4.
workflows:
version: 2
build:
- aws-ecr/build_and_push_image:
account-url: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com
repo: '${AWS_RESOURCE_NAME_PREFIX}'
region: ${AWS_DEFAULT_REGION}
tag: '${CIRCLE_SHA1}'
This is part of my CircleCI setup.
This works totally fine when version is below 0.0.4
Refer to this issue#41, I changed orb job name to aws-ecr/build-and-push-image
when I try to use circleci/[email protected]
However this type check error always showed up when I try to build.
What happened?
Build success like v0.0.4.
the URL can be derived directly from the region and the account id. The orb, already having requesting the region as a parameter, should simply request the account_id and construct the url internally form that information.
- aws-ecr/build_and_push_image:
account-id: account_id #use account_id not explicit URL
region: region
...
The above is enough information to construct the url.
https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html
This makes the Orb interface easier and more DRY by limiting errors and redundant information.
4.0.1
I tried to add pre-steps to the orb job as per the 2.1 documentation for jobs but the schema validation fails.
workflows:
build:
jobs:
- test
- build-bundle
# https://github.com/CircleCI-Public/aws-ecr-orb
- aws-ecr/build_and_push_image:
repo: "my-container-registry"
region: AWS_DEFAULT_REGION
tag: "${CIRCLE_SHA1}"
aws-access-key-id: AWS_ECR_USER_KEY
aws-secret-access-key: AWS_ECR_USER_SECRET
requires:
- test
- build-bundle
pre-steps:
- attach_workspace:
at: bundle
I get this pretty unhelpful error
# ERROR IN CONFIG FILE:
# [#/jobs/aws-ecr~1build_and_push_image] only 1 subschema matches out of 2
# 1. [#/jobs/aws-ecr~01build_and_push_image/steps/0] 0 subschemas matched instead of one
# | 1. [#/jobs/aws-ecr~001build_and_push_image/steps/0] Input not a valid enum value
# | | Steps without arguments can be called as strings
# | | enum:
# | | - checkout
# | | - setup_remote_docker
# | | - add_ssh_keys
# | 2. [#/jobs/aws-ecr~001build_and_push_image/steps/0] extraneous key [at] is not permitted
# | | Permitted keys:
# | | - persist_to_workspace
# | | - save_cache
# | | - run
# | | - checkout
# | | - attach_workspace
# | | - store_test_results
# | | - restore_cache
# | | - store_artifacts
# | | - add_ssh_keys
# | | - deploy
# | | - setup_remote_docker
# | | Passed keys:
# | | []
If pre-steps would work I could actually use the orb job to attach some stuff from another job before building the docker image.
Docker 19.03 now supports buildkit which can be activated by setting the environment variable DOCKER_BUILDKIT=1
before running the docker build
command. https://docs.docker.com/develop/develop-images/build_enhancements/
I have not found a way to do this so I am assuming it is not possible with the current orb. Correct me if I'm wrong please :)
Buildkit significantly decreases build times and allows for better secret management via the --secret command line argument
3.0.0
Currently most of the parts of this repo( the build and push image command was of interest to me) use the default AWS keys env vars( AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
).
For some repos we have multiple AWS keys( say one for pushing the image and one for some CI/CD tools) and we push to multiple AWS ECR repos.
It would be awesome if all the components would support given keys.
Right now we implemented a custom copy/paste jobs that looks like:
- Build and push image:
repo_name: myRepo
aws_account: "22222222222222"
aws_access_key: FANCY_AWS_ACCESS_KEY_ID_1
aws_secret_key: FANCY_AWS_SECRET_ACCESS_KEY_2
3.0.0
Currently images can only have 1 tag as per your example due to how the job is written:
version: 2.1
orbs:
aws-ecr: circleci/[email protected]
workflows:
complete_build_and_push:
jobs:
- aws-ecr/build_and_push_image:
aws-access-key-id: ACCESS_KEY_ID_ENV_VAR_NAME
aws-secret-access-key: SECRET_ACCESS_KEY_ENV_VAR_NAME
region: AWS_REGION_ENV_VAR_NAME
account-url: AWS_ECR_ACCOUNT_URL_ENV_VAR_NAME
repo: myECRRepository
# ECR image tag, defaults to "latest"
tag: ${CIRCLE_BRANCH}-${CIRCLE_BUILD_NUM}
There is a desire for an image to have multiple tags, in our case both the ${CIRCLE_BRANCH}-${CIRCLE_BUILD_NUM}
, the ${CIRCLE_SHA1}
, and say latest
.
version: 2.1
orbs:
aws-ecr: circleci/[email protected]
workflows:
complete_build_and_push:
jobs:
- aws-ecr/build_and_push_image:
aws-access-key-id: ACCESS_KEY_ID_ENV_VAR_NAME
aws-secret-access-key: SECRET_ACCESS_KEY_ENV_VAR_NAME
region: AWS_REGION_ENV_VAR_NAME
account-url: AWS_ECR_ACCOUNT_URL_ENV_VAR_NAME
repo: myECRRepository
tag: [ ${CIRCLE_BRANCH}-${CIRCLE_BUILD_NUM}, ${CIRCLE_SHA1}, "latest"]
This is currently blocked by https://ideas.circleci.com/ideas/CCI-I-701 which asks for list parameters to be enabled.
Are there any workaround for this?
workflows:
version: 2
build_and_push_image:
jobs:
- aws-ecr/build-and-push-image:
account-url: AWS_ECR_ACCOUNT_URL
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
context: myContext
create-repo: true
dockerfile: Dockerfile
path: ./dir
profile-name: circleci
region: AWS_REGION
repo: AWS_REPO
tag: latest
i am getting Missing required argument(s): repo. what i did wrong?
To work :)
Change the default region environment variable value to DEFAULT_AWS_REGION
from AWS_REGION
The orbs current default doesn't match up with the AWS CLI or the AWS CLI orb, meaning when using them both together, one of the default parameters has to be overridden (or they both need to be set).
It looks like this change was made a while back for the CLI orb CircleCI-Public/aws-cli-orb#2
6.7.1
command: build-and-push-image
1) workflow A
parameters : account-url: A_AWS_ACCOUNT_URL
RUN : OK
2) workflow B
parameters : account-url: B_AWS_ACCOUNT_URL
RUN:
Seems to be an issue with multiple workflows and different account urls
docker login stays maybe in the executor cache: because at the beginning of workflow B: the /home/circleci/.docker/config.json contains the docker login of workflow A.
at the very least, basic validation + automated orb publishing
I config the aws-ecr like in the demo, and i get that error
#!/bin/sh -eo pipefail
I may be missing how to accomplish this specific case, but this came up trying to build and push a docker image to ECR which requires pulling from a private docker image. In my existing CircleCI flow, I log into Docker prior to attempting to build the image.
This could be more broadly solved by adding lifecycle hooks, similar to the Docker orb:
https://circleci.com/orbs/registry/orb/circleci/docker-publish
I'm going to take a stab at implementing it tonight and get a PR up in case this is something others find useful!
6.7.0
The ecr/build-and-push-image is called with minimal parameters (repo, create-repo, tag, and region). A .python-version
file specifies the python version to use during the execution of the Dockerfile commands (which is included in the base image). The Install AWS CLI
step fails with the following message:
#!/bin/bash -eo pipefail
export PIP=$(which pip pip3 | head -1)
if [[ -n $PIP ]]; then
if which sudo > /dev/null; then
sudo $PIP install awscli --upgrade
else
# This installs the AWS CLI to ~/.local/bin. Make sure that ~/.local/bin is in your $PATH.
$PIP install awscli --upgrade --user
fi
elif [[ $(which unzip curl | wc -l) -eq 2 ]]; then
cd
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
if which sudo > /dev/null; then
sudo ~/awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
else
# This installs the AWS CLI to the default location (~/.local/lib/aws) and create a symbolic link (symlink) at ~/bin/aws. Make sure that ~/bin is in your $PATH.
awscli-bundle/install -b ~/bin/aws
fi
rm -rf awscli-bundle*
cd -
else
echo "Unable to install AWS CLI. Please install pip."
exit 1
fi
pyenv: version `2.7.17' is not installed (set by /home/circleci/project/.python-version)
Exited with code exit status 1
The .python-version in the local repo should be masked or ignored by the aws-cli installation step since that is completely independent of build step which requires it. I am able to successfully build the image by removing the .python-version from the repo and creating it in the Dockerfile, but this is not ideal.
6.1.0
account-url
is only exposed as an environment variable.
account-url
is exposed as an environment variable and a static string.
6.0.0
Right now by using aws-ecr/build-and-push-image
as command, I got the following error in "Build docker image" step:
$ #!/bin/bash -eo pipefail
docker build \
\
-f ./docker/web/Dockerfile \
-t $AWS_ECR_ACCOUNT_URL/qa-repository:${CIRCLE_SHA1} \
./docker/web
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Exited with code 1
https://circleci.com/gh/gitenter/gitenter/645
By adding setup_remote_docker
things will pass with no problem.
qa-deploy:
docker:
- image: 'circleci/python:2.7'
steps:
- checkout
- setup_remote_docker:
docker_layer_caching: true
- aws-ecr/build-and-push-image:
...
https://circleci.com/gh/gitenter/gitenter/647
It is kind of annoying for command user to setup_remote_docker
themselves. It will be better if that step can just be included in the orb itself.
(Not sure if it relates to the docker image I am using. By using circleci/python:2.7
things goes fine, but if I use circleci/python:3.7.1
I fail the "Log into Amazon ECR" step.
6.3.0
Build fails as version tagged locally is different than version trying to be pushed.
Tag should be the same
.circle/config.yml
orbs:
aws-ecr: circleci/[email protected]
version: 2.1
workflows:
# Build and push to ECR on builds to master
build_and_push_image:
jobs:
- aws-ecr/build-and-push-image:
account-url: AWS_ACCOUNT_URL
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
region: AWS_REGION
repo: node
tag: $(date -u +"%Y-%m-%dT%H%M%SZ") # ISO 8601 date format
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.