sjednac / sbt-ecr Goto Github PK

View Code? Open in Web Editor NEW

53.0 7.0 25.0 94 KB

An SBT plugin for managing Docker images within Amazon ECR.

License: Other

Scala 100.00%

aws aws-ecr amazon-ecr sbt docker sbt-plugin

sbt-ecr's Introduction

Hi there 👋

sbt-ecr's People

Contributors

Stargazers

Watchers

sbt-ecr's Issues

Incompatible with latest AWS SDK

I'm using the latest aws-java-sdk (1.11.289) and it appears sbt-ecr is only compatible with sdks around version 1.11.33.

Following the examples in the README I hit a couple errors.

Here's the subset of my build.sbt pertaining to sbt-ecr:


region           in Ecr := Region.getRegion(Regions.US_EAST_1)
repositoryName   in Ecr := "homebay"
localDockerImage in Ecr := (packageName in Docker).value + ":" + (version in Docker).value

// Create the repository before authentication takes place (optional)
login in Ecr := ((login in Ecr) dependsOn (createRepository in Ecr)).value

// Authenticate and publish a local Docker image before pushing to ECR
push in Ecr := ((push in Ecr) dependsOn (publishLocal in Docker, login in Ecr)).value

The first is the createRepository task fails to instantiate com.amazonaws.services.ecr.AmazonECRClient:

[homebay] $ reload
[info] Loading global plugins from /home/jsimpson/.sbt/0.13/plugins
[info] Loading project definition from /home/jsimpson/workspace/homebay/project
[info] Set current project to homebay (in build file:/home/jsimpson/workspace/homebay/)
[homebay] $ feeds/ecr:push
[info] Wrote /home/jsimpson/workspace/homebay/feeds/target/scala-2.12/feeds_2.12-1.0.0.pom
[info] Wrote /home/jsimpson/workspace/homebay/target/scala-2.12/homebay_2.12-c9f759ef98a419ade612c71f1663c5eb48b33a9f-SNAPSHOT.pom
[info] Sending build context to Docker daemon  223.4MB
[info] Step 1/7 : FROM openjdk:latest
[info]  ---> db77212ffe05
[info] Step 2/7 : LABEL MAINTAINER "Jeff Simpson <[email protected]>"
[info]  ---> Using cache
[info]  ---> 99b1f1d6ca3a
[info] Step 3/7 : WORKDIR /.
[info]  ---> Using cache
[info]  ---> 46528284043e
[info] Step 4/7 : ADD --chown=daemon:daemon . /.
[info]  ---> Using cache
[info]  ---> f3e17eb90f59
[info] Step 5/7 : USER daemon
[info]  ---> Using cache
[info]  ---> b32c0ead8e64
[info] Step 6/7 : ENTRYPOINT bin/feeds
[info]  ---> Using cache
[info]  ---> b7e7906a8bbc
[info] Step 7/7 : CMD
[info]  ---> Using cache
[info]  ---> 5a109b8f36ca
[info] Successfully built 5a109b8f36ca
[info] Successfully tagged homebay-feeds:1.0.0
[info] Built image homebay-feeds:1.0.0
[trace] Stack trace suppressed: run last feeds/ecr:createRepository for the full output.
[error] (feeds/ecr:createRepository) Couldn't instantiate instance of class com.amazonaws.services.ecr.AmazonECRClient
[error] Total time: 3 s, completed Mar 6, 2018 9:31:57 PM

I also tried changing the login task to not depend on createRepository and then get a different error more telling

[homebay] $ feeds/ecr:push
[info] Wrote /home/jsimpson/workspace/homebay/feeds/target/scala-2.12/feeds_2.12-1.0.0.pom
[info] Wrote /home/jsimpson/workspace/homebay/target/scala-2.12/homebay_2.12-c9f759ef98a419ade612c71f1663c5eb48b33a9f-SNAPSHOT.pom
[info] Sending build context to Docker daemon  223.4MB
[info] Step 1/7 : FROM openjdk:latest
[info]  ---> db77212ffe05
[info] Step 2/7 : LABEL MAINTAINER "Jeff Simpson <[email protected]>"
[info]  ---> Using cache
[info]  ---> 99b1f1d6ca3a
[info] Step 3/7 : WORKDIR /.
[info]  ---> Using cache
[info]  ---> 46528284043e
[info] Step 4/7 : ADD --chown=daemon:daemon . /.
[info]  ---> Using cache
[info]  ---> f3e17eb90f59
[info] Step 5/7 : USER daemon
[info]  ---> Using cache
[info]  ---> b32c0ead8e64
[info] Step 6/7 : ENTRYPOINT bin/feeds
[info]  ---> Using cache
[info]  ---> b7e7906a8bbc
[info] Step 7/7 : CMD
[info]  ---> Using cache
[info]  ---> 5a109b8f36ca
[info] Successfully built 5a109b8f36ca
[info] Successfully tagged homebay-feeds:1.0.0
[info] Built image homebay-feeds:1.0.0
[trace] Stack trace suppressed: run last feeds/ecr:login for the full output.
[error] (feeds/ecr:login) java.lang.NoSuchMethodError: com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.getCallerIdentity(Lcom/amazonaws/services/securitytoken/model/GetCallerIdentityRequest;)Lcom/amazonaws/services/securitytoken/model/GetCallerIdentityResult;
[error] Total time: 3 s, completed Mar 6, 2018 9:33:31 PM

Any chance we could get a new release updated against the latest API?

Artifact no longer found

[error] Error downloading com.mintbeans:sbt-ecr;sbtVersion=1.0;scalaVersion=2.12:0.16.0
[error]   Not found
[error]   Not found
[error]   not found: https://repo1.maven.org/maven2/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom
[error]   not found: /Users/fabienfoerster/.ivy2/local/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml
[error]   not found: https://davidmweber.github.io/flyway-sbt.repo/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml.sha1
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.typesafe.com/typesafe/ivy-releases/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml.sha1
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom.sha1

Missing resolver information in documentation

I only found information about the resolver in your blog post:

resolvers += Resolver.url("bintray-sbilinski", url("http://dl.bintray.com/sbilinski/maven"))(Resolver.ivyStylePatterns)

http://mintbeans.com/sbt-amazon-ecr/

I'll create a PR later to update the README to something similar as you've written in your blog post.

Assume AWS role when executing AWS API

The shared role doesn't get assumed when executing the following:

sbt -DAWS_DEFAULT_PROFILE=shared ecr:createRepository

I have the following in my aws files:

[default]
aws_access_key_id = xxx
aws_secret_access_key = yyy

[shared]
aws_access_key_id = xxx
aws_secret_access_key = yyy

[default]
output = json
region = us-west-2

[profile shared]
role_arn = zzz
source_profile = default
output = json
region = us-west-2

I see this line in AWS.scala:

...
new ProfileCredentialsProvider(sys.env.getOrElse("AWS_DEFAULT_PROFILE", "default")),
...

Any pointers would be be helpful! I see Thank you!

Support image tag mutability property

Currently, there is no way to set the image tag mutability property on the CreateRepositoryRequest when creating a new repository.

Cannot push image

I'm using version 0.3.0 of the sbt-ecr plugin, but I'm not able to push the image. Two other commands ecr:login and ecr:createRepository work OK, but push apparently does nothing.

I am using Docker 1.13 on a Mac, and my suspicion that tagging https://github.com/sbilinski/sbt-ecr/blob/master/src/main/scala/sbtecr/EcrPlugin.scala#L57) step does not work. I am not a Scala person, but if I understand this code correctly, it should add tag to the the image with the registry name prepended.

When I run docker images after ecr:push image is not tagged:

$ docker images
REPOSITORY                                          TAG                 IMAGE ID            CREATED             SIZE
myimage                                             latest              75e222321012        4 minutes ago       257 MB
<aws_account>.dkr.ecr.us-east-1.amazonaws.com/java  latest              d5c4c8395ef9        3 months ago        174 MB

This is my configuration:

name := "myimage"
version := "latest"

packageName in Docker := "myimage"
version     in Docker := version.value

dockerBaseImage := "<aws_account>.dkr.ecr.us-east-1.amazonaws.com/java:latest"
dockerExposedPorts := Seq(9000)

import com.amazonaws.regions.{Region, Regions}

region           in ecr := Region.getRegion(Regions.US_EAST_1)
repositoryName   in ecr := (packageName in Docker).value
localDockerImage in ecr := (packageName in Docker).value + ":" + (version in Docker).value

addCommandAlias("publish", ";clean;compile;docker:publishLocal;ecr:createRepository;ecr:login;ecr:push")

And docker version:

$ docker version                                                                                                                                                                                    U feature/build_with_jenkins:exco-corp-web/. 12:47:20
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Wed Jan 18 16:20:26 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Wed Jan 18 16:20:26 2017
 OS/Arch:      linux/amd64
 Experimental: true

link to sbt shared plugin repository

Can you please link your excellent plugin to the shared sbt plugins repository on bintray?
This would make your sbt-ecr plugin more easily accessible for users behind a corporate Nexus/artifactory gateway.

Since you are already publishing on bintray, it is just a small effort.
Check the documentation for details: http://www.scala-sbt.org/0.13/docs/Bintray-For-Plugins.html#Linking+your+package+to+the+sbt+organization

ECR lifecycle policy support

Provided by #34.

Support "push" to multiple tags within the same repository

Check comments in #7.

`docker login`'s `-e` no longer supported

I installed the latest beta of Docker for Mac. Apparently docker login does not support the -e check anymore.

Now the sbt task docker:login does always fail.

I think the fix would be to remove it from https://github.com/sbilinski/sbt-ecr/blob/master/src/main/scala/sbtecr/EcrPlugin.scala#L41.

denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

Hi,

I am getting the following error:

denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

Before the push, I can see:

[info] AWS account id: 07579032XXXX
The push refers to repository [07579032XXXX.dkr.ecr.ap-south-1.amazonaws.com/XXXX-service]

Which seems to suggest that the login was succesful.

Any ideas what may be the issue? (full log is attached)

Jenkins-error.txt

Support ECS / CodeBuild provided credentials

This can be fixed in a few different ways:

Allow the provider chain to be specified as an sbt setting. Onus on is on the user.
Update current provider chain to use EC2ContainerCredentialsProviderWrapper which wraps the current InstanceProfileCredentialsProvider and supports container credentials. This is what's done in the aws DefaultAWSCredentialsProviderChain
Use the standard aws DefaultAWSCredentialsProviderChain. It seems the only thing that's different in this project is the support for AWS_DEFAULT_PROFILE. This is not something that's supported in any AWS sdk. This was used for a time by the CLI but has since been deprecated in favor of AWS_PROFILE https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html which the DefaultAWSCredentialsProviderChain already handles.

Allow profile to be set via system property as an alternative

Although, it's nice to execute:

AWS_DEFAULT_PROFILE="sandbox" sbt ecr:createRepository

it'd be nice to have an alternative to execute:

sbt -Daws.profile=sandbox ecr:createRepository

This also has a nice property that the build can be configured via the initialize sbt task which prevents for users to specify the profile by default if not found.

initialize ~= { _ =>
  System.setProperty("aws.profile", sys.props.getOrElse("aws.profile", "sandbox"))
}

Create a repository policy

Add ability to set a repository policy when creating a new repository.

Expose ECR repository domain as a task key

Provided by #32.

Support SBT 1.x

Update sbt-ecr to SBT 1.0.2

Pushing to Region CN_North_1 doesn't work

I tried to automate an upload to AWS China (Region CN_North_1), however the domain
def domain(region: Region, accountId: String) = s"${accountId}.dkr.ecr.${region}.amazonaws.com"
doesn't take into account that AWS' domain in China is amazonaws.com.CN instead of only amazonaws.com.
The debug output from the AWS SDK displays the right URL during the build process:

INFORMATION: {ecr, cn-north-1} was not found in region metadata, trying to construct an endpoint using the standard pattern for this region: 'ecr.cn-north-1.amazonaws.com.cn'.
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
Error response from daemon: Get https://************.dkr.ecr.cn-north-1.amazonaws.com/v2/: dial tcp: lookup ************.dkr.ecr.cn-north-1.amazonaws.com on 192.168.65.1:53: no such host

The AWS_DEFAULT_PROFILE environment variable has been deprecated in favour of AWS_PROFILE

This plugin is using the deprecated AWS_DEFAULT_PROFILE environment variable in place of the AWS_PROFILE.

The usage of this environment variable is documented here: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

Worth noting that the behaviour of the ProfileCredentialsProvider class resolves the AWS_PROFILE environment variable and used default as a fallback.

ECR version is always set to latest

The ecr image is always tagged as latest. Please consider adding a version Setting to the plugin which will be included when tagging the docker image.

Support submodules

The plugin does support submodules.

The following config

lazy val core = (project in file("core")).
  settings(
    scalaVersion := "2.11.8",
    version := "1.0",
    name := "sbt-ecr-test",
    packageName in Docker := "helloworld-ecr",
    version     in Docker := version.value,
    region           in ecr := Region.getRegion(Regions.US_WEST_2),
    repositoryName   in ecr := (packageName in Docker).value,
    localDockerImage in ecr := (packageName in Docker).value + ":" + (version in Docker).value,
    push in ecr <<= (push in ecr) dependsOn (publishLocal in Docker)
  ).enablePlugins(JavaAppPackaging, DockerPlugin)

Will raise the following error while importing sbt-ecr.

[error] References to undefined settings:
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:localDockerImage ((sbtecr.EcrPlugin) EcrPlugin.scala:29)
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:createRepository ((sbtecr.EcrPlugin) EcrPlugin.scala:33)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:createRepository ((sbtecr.EcrPlugin) EcrPlugin.scala:33)

I think this can be fixed by by dropping the following line and enabling the plugin via .enableplugins just like you have to for sbt-native-packager

override def trigger = allRequirements

Thanks for the helpful plugin.

After updating from 0.13.0 to 0.14.0 sbt fails with Reference to undefined setting

After I bump the version from 0.13.0 to 0.14.0 the plugin fails at sbt load. The error is

[error] Reference to undefined setting:
[error]
[error]   app / Ecr / repositoryLifecyclePolicyText from app / Ecr / createRepository ((sbtecr.EcrPlugin.tasks) EcrPlugin.scala:43)
[error]
[error] 	at sbt.internal.util.Init.Uninitialized(Settings.scala:360)
[error] 	at sbt.internal.util.Init.Uninitialized$(Settings.scala:347)
[error] 	at sbt.Def$.Uninitialized(Def.scala:20)
[error] 	at sbt.internal.util.Init.delegate(Settings.scala:260)
[error] 	at sbt.internal.util.Init.delegate$(Settings.scala:236)
[error] 	at sbt.Def$.delegate(Def.scala:20)
[error] 	at sbt.internal.util.Init.compiled(Settings.scala:194)
[error] 	at sbt.internal.util.Init.compiled$(Settings.scala:182)
[error] 	at sbt.Def$.compiled(Def.scala:20)
[...]

This would get rid of this annoying console spam caused by AWS SDK (that has been finally fixed in SDK):

Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.ersion 0s
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.

sjednac / sbt-ecr Goto Github PK

sbt-ecr's Introduction

Hi there 👋

sbt-ecr's People

Contributors

Stargazers

Watchers

Forkers

sbt-ecr's Issues

Recommend Projects

Recommend Topics

Recommend Org