hashicorp / packer-plugin-docker Goto Github PK

View Code? Open in Web Editor NEW

30.0 12.0 25.0 945 KB

Packer plugin for Docker Builder

Home Page: https://www.packer.io/docs/builders/docker

License: Mozilla Public License 2.0

Go 89.07% HCL 2.09% Makefile 0.78% MDX 4.76% Shell 3.30%

packer-plugin packer docker

packer-plugin-docker's Introduction

Packer Plugin Docker

The Docker multi-component plugin can be used with HashiCorp Packer to create custom images. For the full list of available features for this plugin see docs.

Installation

Using pre-built releases

Using the `packer init` command

Starting from version 1.7, Packer supports a new packer init command allowing automatic installation of Packer plugins. Read the Packer documentation for more information.

To install this plugin, copy and paste this code into your Packer configuration . Then, run packer init.

packer {
  required_plugins {
    docker = {
      version = ">= 1.0.9"
      source  = "github.com/hashicorp/docker"
    }
  }
}

Manual installation

You can find pre-built binary releases of the plugin here. Once you have downloaded the latest archive corresponding to your target OS, uncompress it to retrieve the plugin binary file corresponding to your platform. To install the plugin, please follow the Packer documentation on installing a plugin.

From Sources

If you prefer to build the plugin from sources, clone the GitHub repository locally and run the command go build from the root directory. Upon successful compilation, a packer-plugin-docker plugin binary file can be found in the root directory. To install the compiled plugin, please follow the official Packer documentation on installing a plugin.

Configuration

For more information on how to configure the plugin, please read the documentation located in the docs/ directory.

Contributing

See CONTRIBUTING.md for best practices and instructions on contributing to Docker Plugin.

packer-plugin-docker's People

Contributors

Stargazers

Watchers

packer-plugin-docker's Issues

Packer should remove /packer-files mount point

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

Currently the Packer Docker builder mounts a host directory with permissions 0700 to the temporary Docker container as /packer-files. When the Docker image is committed, the /packer-files mount point persists. I am requesting that the builder be changed to remove the mount point (as long as the builder had to create it) before the image is committed.

Use Case(s)

I have a simple example in simple1/ @ my bruno-packer github repo. Demonstrating with:

Packer 1.8.4
Docker 20.10.12
Host OS: Ubuntu 20

$ cd simple1
$ packer build simple1.pkr.hcl
docker.simple1: output will be in this color.

==> docker.simple1: Creating a temporary directory for sharing data...
==> docker.simple1: Pulling Docker image: ubuntu:latest
    docker.simple1: latest: Pulling from library/ubuntu
    docker.simple1: Digest: sha256:27cb6e6ccef575a4698b66f5de06c7ecd61589132d5a91d098f7f3f9285415a9
    docker.simple1: Status: Image is up to date for ubuntu:latest
    docker.simple1: docker.io/library/ubuntu:latest
==> docker.simple1: Starting docker container...
    docker.simple1: Run command: docker run -v /home/ubuntu/.config/packer/tmp3941861951:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:latest
    docker.simple1: Container ID: 7bc41d246591646c0d6c862fe33cce329f695b3f42c797f92dd76e9fe6a6222b
==> docker.simple1: Using docker communicator to connect: 172.17.0.2
==> docker.simple1: Provisioning with shell script: /tmp/packer-shell999041850
==> docker.simple1: Committing the container
    docker.simple1: Image ID: sha256:1b5a62e69852bced6035ca8de430530787eb68e95ca6dea9c5e6f6cf58f300b9
==> docker.simple1: Killing the container: 7bc41d246591646c0d6c862fe33cce329f695b3f42c797f92dd76e9fe6a6222b
==> docker.simple1: Running post-processor:  (type docker-tag)
    docker.simple1 (docker-tag): Tagging image: sha256:1b5a62e69852bced6035ca8de430530787eb68e95ca6dea9c5e6f6cf58f300b9
    docker.simple1 (docker-tag): Repository: simple1:latest
Build 'docker.simple1' finished after 2 seconds 922 milliseconds.

==> Wait completed after 2 seconds 922 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.simple1: Imported Docker image: sha256:1b5a62e69852bced6035ca8de430530787eb68e95ca6dea9c5e6f6cf58f300b9
--> docker.simple1: Imported Docker image: simple1:latest with tags simple1:latest
$ docker run -it --rm simple1:latest
# ls -ld /packer-files
drwxr-xr-x 2 root root 4096 Dec 22 12:07 /packer-files
# find /packer-files
/packer-files
#

This isn't a major issue and no files are left under /packer-files since it was just a mount point but there's no need for the directory in the committed image and it's just baggage that isn't needed.

Potential configuration

n/a

Potential References

I posed this issue as a question in the community but got no responses.

This might be more of a Docker issue but I thought I would start with the Packer Docker builder.

When provisioning a docker container, it is not clear how to use the instance_id parameter in the provisioner(s)

Overview of the Issue

I have a packer file containing the following:

[....]
source "docker" "Test" {
    image        = "centos:7"
    export_path  = "test.tar"
}
[....]
build {
  sources = ["source.docker.Test"]
  provisioner "shell" {
    inline = ["echo 'proxy=http://<proxy_url>' >> /etc/yum.conf", "rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7", "yum install -y python3"]
  }

  provisioner "ansible" {
    extra_arguments = [
      "-e", "proxy_url='http://<proxy_url>'",
      "-e", "ansible_connection=docker"
    ]
    playbook_file   = "playbooks/GoldenImage.yml"
    user            = "root"
  }
}

Ansible fails the first time that tries to connect to the docker container (when doing the initial host scan, called "facts gathering" in Ansible). And I suspect is because the contents of the inventory file are like

default ansible_host=127.0.0.1 ansible_user=root ansible_port=42779

By setting the host_alias parameter in ansible provisioner I can work around this situation. However, I do not get to see how can I reference the builder.instance_id variable from the config file.

Plugin and Packer version

packer version: 1.7.4
I do not know how to get the plugin versions

Can't mount volume to docker container when volume is user variable.

This issue was originally opened by @CollinLeishman in hashicorp/packer#11163 and has been migrated to this repository. The original issue description is below.

Overview of the Issue

Trying to mount current working directory(ex:/home/user/testing/ on my machine to /home/user/testing/ on docker container) fails because the volume variable is not rendered for the container. It tries you mount /home/user/testing/ to {{ user workspace }}/build/repo on the docker container.

Reproduction Steps

Reproduced with Docker version 20.10.7, build f0df350 and 17.12.1-ce, build 7390fc6, so install either of those.

PACKER_LOG=1 packer build -var 'workspace=/home/user/testing/' testing.json

Packer version

1.7.3

Simplified Packer Buildfile

{
  "builders": [
    {
      "commit": true,
      "image": "docker.io/centos:centos7.6.1810",
      "type": "docker",
      "volumes": {
        "{{ user `workspace` }}/build/repo": "{{ user `workspace` }}/build/repo"
      }
    }
  ],
  "variables": {
    "version": "12.17.1"
  }
}

Operating system and Environment details

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Linux removed_hostname 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux

Log Fragments and crash.log files

==> docker: Starting docker container...
    docker: Run command: docker run -v /home/user/packer_bug_report/build/repo:{{ user `workspace` }}/build/repo -v /home/user/.packer.d/tmp028037932:/packer-files -d -i -t --entrypoint=/bin/sh -- docker.io/centos:centos7.6.1810
2021/07/22 10:12:15 packer-builder-docker plugin: Starting container with args: [run -v /home/user/packer_bug_report/build/repo:{{ user `workspace` }}/build/repo -v /home/user/.packer.d/tmp028037932:/packer-files -d -i -t --entrypoint=/bin/sh -- docker.io/centos:centos7.6.1810]
2021/07/22 10:12:15 packer-builder-docker plugin: Waiting for container to finish starting
==> docker: Stderr: docker: Error response from daemon: invalid volume specification: '/home/user/packer_bug_report/build/repo:{{ user `workspace` }}/build/repo': invalid mount config for type "bind": invalid mount path: '{{ user `workspace` }}/build/repo' mount path must be absolute.
==> docker: Error running container: Docker exited with a non-zero exit status.
==> docker: See 'docker run --help'.
==> docker:
2021/07/22 10:12:16 [INFO] (telemetry) ending docker
Stderr: docker: Error response from daemon: invalid volume specification: '/home/user/packer_bug_report/build/repo:{{ user `workspace` }}/build/repo': invalid mount config for type "bind": invalid mount path: '{{ user `workspace` }}/build/repo' mount path must be absolute.
See 'docker run --help'.

Push docker image to ECR public gallery

This issue was originally opened by @Laakso as hashicorp/packer#10948. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Description

Since AWS released their public container registry it would be nice that the packer image would be pushed there as well.

Use Case(s)

Push packer container image to ECR public container registry.

builder/docker: upload files with `-a` flag when upstream bug is fixed

This issue was originally opened by @mwhooker as hashicorp/packer#5409. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

We should set ownership of files based on the container user. In theory, this should be doable with the -a flag to docker cp, but there's a bug that prevents that. When moby/moby#34142 is fixed, let's change docker cp to be docker cp -a

Docker builder documentation is incorrect (fix_upload_owner)

This issue was originally opened by @boblechat in hashicorp/packer#11187 and has been migrated to this repository. The original issue description is below.

Overview of the Issue

Building a docker image from a base image running with a non root user always causes a "Permission denied" in shell provisioners.
Documentation clearly states that the fix_upload_owner is set to true by defaut, which is incorrect. You have to set it explicitely!
Also, could you clarify what "the owner will depend on the version of docker installed in the system" means? That's gibbersih to me.

Reproduction Steps

To make it fail:

packer build base.pkr.hcl
packer build agent.pkr.hcl

To make it work;

packer build base.pkr.hcl
uncomment fix_upload_owner = true in agent.pkr.hcl
packer build agent.pkr.hcl

Packer version

1.7.4

Simplified Packer Buildfile

gist is here

Operating system and Environment details

Irrelevant here

Log Fragments and crash.log files

When it fails:

==> agent.docker.alm: Using docker communicator to connect: 172.17.0.2
==> agent.docker.alm: Provisioning with shell script: C:\Users\me\AppData\Local\Temp\packer-shell381356875
==> agent.docker.alm: sh: 1: /tmp/script_120.sh: Permission denied
==> agent.docker.alm: Provisioning step had errors: Running the cleanup provisioner, if present...

When it works:

==> agent.docker.alm: Using docker communicator to connect: 172.17.0.2
==> agent.docker.alm: Provisioning with shell script: C:\Users\me\AppData\Local\Temp\packer-shell892913999
    agent.docker.alm: Hello from Hell!
==> agent.docker.alm: Committing the container

Updating tag with the build variable `ImageSha256` fails

Overview of the Issue

I would like to tag the image with the build variable ImageSha256, however this fails.

Reproduction Steps

packer build the buildfile below

Plugin and Packer version

Packer: 1.7.2

Simplified Packer Buildfile

{
  "builders": [{
    "type": "docker",
    "image": "centos:7",
    "commit": true
  }],
  "post-processors": [{
    "type": "docker-tag",
    "repository": "centos",
    "tags": ["{{ build `ImageSha256` }}"]
  }]
}

Operating system and Environment details

MacOS Big Sur 11.4
Docker version 20.10.6, build 370c289
Docker Desktop 3.3.3

Log Fragments and crash.log files

2021/06/24 09:54:55 [INFO] Packer version: 1.7.2 [go1.16.3 darwin amd64]
2021/06/24 09:54:55 [TRACE] discovering plugins in /usr/local/bin
2021/06/24 09:54:55 [DEBUG] Discovered plugin: windows-update = /usr/local/bin/packer-provisioner-windows-update
2021/06/24 09:54:55 using external provisioners [windows-update]
2021/06/24 09:54:55 [TRACE] discovering plugins in /Users/cr2p/.packer.d/plugins
2021/06/24 09:54:55 [TRACE] discovering plugins in .
2021/06/24 09:54:55 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2021/06/24 09:54:55 [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 [WARN] Config file doesn't exist: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 [INFO] Setting cache directory: /Users/cr2p/git/issues_filed/packer_docker_tag/packer_cache
2021/06/24 09:54:55 [TRACE] Starting internal plugin packer-builder-docker
2021/06/24 09:54:55 Starting plugin: /usr/local/bin/packer []string{"/usr/local/bin/packer", "plugin", "packer-builder-docker"}
2021/06/24 09:54:55 Waiting for RPC address for: /usr/local/bin/packer
2021/06/24 09:54:55 packer-builder-docker plugin: [INFO] Packer version: 1.7.2 [go1.16.3 darwin amd64]
2021/06/24 09:54:55 packer-builder-docker plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2021/06/24 09:54:55 packer-builder-docker plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 packer-builder-docker plugin: [WARN] Config file doesn't exist: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 packer-builder-docker plugin: [INFO] Setting cache directory: /Users/cr2p/git/issues_filed/packer_docker_tag/packer_cache
2021/06/24 09:54:55 packer-builder-docker plugin: args: []string{"packer-builder-docker"}
2021/06/24 09:54:55 Received unix RPC address for /usr/local/bin/packer: addr is /var/folders/jg/d47sqv3j2d5_53lgvxntfj5w5xmhz6/T/packer-plugin542462574
2021/06/24 09:54:55 packer-builder-docker plugin: Plugin address: unix /var/folders/jg/d47sqv3j2d5_53lgvxntfj5w5xmhz6/T/packer-plugin542462574
2021/06/24 09:54:55 packer-builder-docker plugin: Waiting for connection...
2021/06/24 09:54:55 packer-builder-docker plugin: Serving a plugin connection...
2021/06/24 09:54:55 [TRACE] Starting internal plugin packer-post-processor-docker-tag
2021/06/24 09:54:55 Starting plugin: /usr/local/bin/packer []string{"/usr/local/bin/packer", "plugin", "packer-post-processor-docker-tag"}
2021/06/24 09:54:55 Waiting for RPC address for: /usr/local/bin/packer
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: [INFO] Packer version: 1.7.2 [go1.16.3 darwin amd64]
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: [WARN] Config file doesn't exist: /Users/cr2p/.packerconfig
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: [INFO] Setting cache directory: /Users/cr2p/git/issues_filed/packer_docker_tag/packer_cache
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: args: []string{"packer-post-processor-docker-tag"}
2021/06/24 09:54:55 Received unix RPC address for /usr/local/bin/packer: addr is /var/folders/jg/d47sqv3j2d5_53lgvxntfj5w5xmhz6/T/packer-plugin195582863
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: Plugin address: unix /var/folders/jg/d47sqv3j2d5_53lgvxntfj5w5xmhz6/T/packer-plugin195582863
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: Waiting for connection...
2021/06/24 09:54:55 packer-post-processor-docker-tag plugin: Serving a plugin connection...
2021/06/24 09:54:55 Preparing build: docker
2021/06/24 09:54:55 Build debug mode: false
2021/06/24 09:54:55 Force build: false
docker: output will be in this color.
2021/06/24 09:54:55 On error:

2021/06/24 09:54:55 Waiting on builds to complete...
2021/06/24 09:54:55 Starting build run: docker
2021/06/24 09:54:55 Running builder: docker
2021/06/24 09:54:55 [INFO] (telemetry) Starting builder docker
2021/06/24 09:54:55 packer-builder-docker plugin: [DEBUG] Docker version: 20.0.0
2021/06/24 09:54:55 packer-builder-docker plugin: [DEBUG] Container will be committed
==> docker: Creating a temporary directory for sharing data...
2021/06/24 09:54:55 packer-builder-docker plugin: Set Packer temp dir to /Users/cr2p/.packer.d/tmp965318901
==> docker: Pulling Docker image: centos:7
2021/06/24 09:54:55 packer-builder-docker plugin: Executing: /usr/local/bin/docker [pull centos:7]
    docker: 7: Pulling from library/centos
    docker: Digest: sha256:0f4ec88e21daf75124b8a9e5ca03c37a5e937e0e108a255d890492430789b60e
    docker: Status: Image is up to date for centos:7
    docker: docker.io/library/centos:7
==> docker: Starting docker container...
    docker: Run command: docker run -v /Users/cr2p/.packer.d/tmp965318901:/packer-files -d -i -t --entrypoint=/bin/sh -- centos:7
2021/06/24 09:54:57 packer-builder-docker plugin: Starting container with args: [run -v /Users/cr2p/.packer.d/tmp965318901:/packer-files -d -i -t --entrypoint=/bin/sh -- centos:7]
2021/06/24 09:54:57 packer-builder-docker plugin: Waiting for container to finish starting
    docker: Container ID: 0846679357b77f78b81f2712c2d61939e4b23e24b383b8680d2183cebf19ba3a
==> docker: Using docker communicator to connect: 172.17.0.2
2021/06/24 09:54:59 packer-builder-docker plugin: Running the provision hook
==> docker: Committing the container
2021/06/24 09:54:59 packer-builder-docker plugin: Committing container with args: [commit 0846679357b77f78b81f2712c2d61939e4b23e24b383b8680d2183cebf19ba3a]
    docker: Image ID: sha256:b5ed38be8b20a28c052a4e53ae734a6d6ecdcc74ce69837860b3590797e5c13f
==> docker: Killing the container: 0846679357b77f78b81f2712c2d61939e4b23e24b383b8680d2183cebf19ba3a
2021/06/24 09:55:00 [INFO] (telemetry) ending docker
2021/06/24 09:55:00 [INFO] (telemetry) Starting post-processor docker-tag
==> docker: Running post-processor: docker-tag
    docker (docker-tag): Tagging image: sha256:b5ed38be8b20a28c052a4e53ae734a6d6ecdcc74ce69837860b3590797e5c13f
    docker (docker-tag): Repository: centos:{{.ImageSha256}}
2021/06/24 09:55:00 packer-post-processor-docker-tag plugin: error: Error tagging image: exit status 1
2021/06/24 09:55:00 packer-post-processor-docker-tag plugin: Stderr: Error parsing reference: "centos:{{.ImageSha256}}" is not a valid repository/tag: invalid reference format
2021/06/24 09:55:00 [INFO] (telemetry) ending docker-tag
2021/06/24 09:55:00 Deleting original artifact for build 'docker'
2021/06/24 09:55:00 packer-builder-docker plugin: Deleting image: sha256:b5ed38be8b20a28c052a4e53ae734a6d6ecdcc74ce69837860b3590797e5c13f

* Post-processor failed: Error tagging image: exit status 1
Stderr: Error parsing reference: "centos:{{.ImageSha256}}" is not a valid repository/tag: invalid reference format

==> Wait completed after 4 seconds 997 milliseconds
2021/06/24 09:55:00 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2021/06/24 09:55:00 machine readable: docker,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Error tagging image: exit status 1\nStderr: Error parsing reference: \"centos:{{.ImageSha256}}\" is not a valid repository/tag: invalid reference format\n"}

* Post-processor failed: Error tagging image: exit status 1
Stderr: Error parsing reference: "centos:{{.ImageSha256}}" is not a valid repository/tag: invalid reference format
==> Builds finished but no artifacts were created.
Build 'docker' errored after 4 seconds 997 milliseconds: 1 error(s) occurred:

2021/06/24 09:55:00 [INFO] (telemetry) Finalizing.
* Post-processor failed: Error tagging image: exit status 1
Stderr: Error parsing reference: "centos:{{.ImageSha256}}" is not a valid repository/tag: invalid reference format


==> Wait completed after 4 seconds 997 milliseconds

==> Some builds didn't complete successfully and had errors:
--> docker: 1 error(s) occurred:

* Post-processor failed: Error tagging image: exit status 1
Stderr: Error parsing reference: "centos:{{.ImageSha256}}" is not a valid repository/tag: invalid reference format


==> Builds finished but no artifacts were created.
2021/06/24 09:55:01 waiting for all plugin processes to complete...
2021/06/24 09:55:01 /usr/local/bin/packer: plugin process exited
2021/06/24 09:55:01 /usr/local/bin/packer: plugin process exited

Docker builder: is it possible to stay logged-in once authenticated on a private registry? [ plugin credentials ]

This issue was originally opened by @boblechat in hashicorp/packer#11180 and has been migrated to this repository. The original issue description is below.

I have a use case where I pull a base image from a private registry where I need to log into.
Then in my build process, I tag and push the newly created image in the same private registry.

When I read the log produced, I see that packer log out automatically after pulling the image which is a bit annoying because I have to log in again in the docker-tag post-processor.

Is there a way to stay connected once authenticated?

Packer docker builder reseting ENTRYPOINT and CMD

This issue was originally opened by @saulfm08 in hashicorp/packer#12529 and has been migrated to this repository. The original issue description is below.

We are using packer to build a container with a parent that includes an entrypoint. In the packer build we "reset" entrypoint with ENTRYPOINT []. Unfortunately the container is being committed with the entrypoint of the parent still in place.

Doing this in a Dockerfile gets rid of the entrypoint from parent:

FROM ep-issue:parent
ENTRYPOINT []
CMD []

While the same in packer version does not:

source "docker" "wordpress" {
  image  = "wordpress:latest"
  commit = true
}

build {
  name = "my-wp"
  sources = [
    "source.docker.wordpress"
  ]

  post-processors {
    post-processor "docker-tag" {
      repository = "${var.docker_registry}/${var.docker_repository}"
      tags       = ["latest"]
    }
    post-processor "docker-push" {
      ecr_login      = true
      login_server   = var.docker_registry
    }
  }
}

docker image inspect wordpress:latest -f '{{ .ContainerConfig.Cmd }}'
[/bin/sh -c #(nop)  CMD ["apache2-foreground"]]

docker image inspect my-wp:latest -f '{{ .ContainerConfig.Cmd }}'
[]


docker image inspect wordpress:latest -f '{{ .ContainerConfig.Entrypoint }}'
[docker-entrypoint.sh]

docker image inspect my-wp:latest -f '{{ .ContainerConfig.Entrypoint }}'
[/bin/sh]

Packer version 1.9.2
Host platform MacOS Ventura

Gist of packer-entrypoint-build.log
Gist of the simple script to reproduce the bug

No meaningful error even in debug mode when Docker Desktop (WSL2) is not running

Overview of the Issue

I'm running Docker Desktop on Windows and execution of docker command in the WSL2 results in an error until Docker Desktop is started (assuming auto start is disabled):

The command 'docker' could not be found in this WSL 2 distro.
We recommend to activate the WSL integration in Docker Desktop settings.

For details about using Docker Desktop with WSL 2, visit:

https://docs.docker.com/go/wsl2/

Until Docker Desktop is started, packer fails in a bit unexpected way:

$ packer build -debug .
Debug mode enabled. Builds will not be parallelized.
docker.openjdk: output will be in this color.

Build 'docker.openjdk' errored after 62 milliseconds 741 microseconds: exit status 1

==> Wait completed after 62 milliseconds 798 microseconds

==> Some builds didn't complete successfully and had errors:
--> docker.openjdk: exit status 1

==> Builds finished but no artifacts were created.

Reproduction Steps

Install WSL2 and Docker Desktop
Make sure docker is not started
Run packer build with a docker source

Plugin and Packer version

Packer v1.8.3

Simplified Packer Buildfile

packer {
  required_plugins {
    docker = {
      version = ">= 1.0.1"
      source  = "github.com/hashicorp/docker"
    }
  }
}

source "docker" "openjdk" {
  image  = "eclipse-temurin:18-jre-jammy"
  commit = true
}

build {
  sources = [
    "source.docker.openjdk"
  ]
  provisioner "shell" {
    inline = ["mkdir /app"]
  }
  post-processor "docker-tag" {
    repository = "test/test1"
    tags       = ["{{isotime `2006-01-02`}}"]
  }
}

Operating system and Environment details

Windows 11, WSL2, Docker Desktop

RFE: Layered container images using Docker Builder

This issue was originally opened by @maxking as hashicorp/packer#6000. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

This feature request follows from hashicorp/packer#5972 for allowing one layer per-provisioner in docker builder, so that we have some opportunity to optimize the container image for sharing layers.

I spent some time hacking on trying to implement a suggestion from @mwhooker but hit a snag when I realized that just changing the HookedProvisioner to something that commits after each provisioner wouldn't work and it would instead result in multiple images, produced after each provisioner ran and sharing no layers.

If I understand correctly, packer runs the base image, provisions it and then commits to an image. To achieve layering, I assume that we'd have to run a provisioner, commit the image and then teardown the entire thing and re-do the StepRun and StepConnect using the image committed in the previous step.

I just wanted to open this issue to discuss what you'd is the best way to achieve this, before I go around making too many intrusive changes?

Simplify Pre-built Binary Release Archive with Unified Filename?

Description

With https://github.com/hashicorp/packer-plugin-docker/releases/tag/v0.0.7 release archive, the unzip result with multiple files and non-unified binary filename (i.e. packer-plugin-docker_v0.0.7_x5.0_linux_amd64):

$ curl -skL -O https://github.com/hashicorp/packer-plugin-docker/releases/download/v0.0.7/packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
$ unzip -o -d . packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
Archive:  packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
  inflating: ./CHANGELOG.md          
  inflating: ./README.md
  inflating: ./packer-plugin-docker_v0.0.7_x5.0_linux_amd64

In case of https://github.com/hashicorp/packer/releases/tag/v1.7.2 itself, the unzip goes into single unified binary filename (i.e. packer):

$ curl -skL -O https://releases.hashicorp.com/packer/1.7.2/packer_1.7.2_linux_amd64.zip
$ unzip -o -d . packer_1.7.2_linux_amd64.zip 
Archive:  packer_1.7.2_linux_amd64.zip
  inflating: ./packer

Shall we simplify our release archive content into single unified binary filename for simpler CI/CD/Scripting integration (i.e. packer-plugin-docker), e.g.

$ curl -skL -O https://github.com/hashicorp/packer-plugin-docker/releases/download/v0.0.7/packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
$ unzip -o -d . packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
Archive:  packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
  inflating: ./packer-plugin-docker

P.S. I temporarily fork the repo and re-zip it for following use case demo:

Use Case(s)

In my GitLab CI pipeline I will install packer if required (see https://github.com/alvistack/vagrant-gitlab-runner/blob/d77a233e61eff4c758b610352be04b55809c8d9d/.gitlab-ci.yml#L8-L17), e.g.

.script-bootstrap-packer: &script-bootstrap-packer
  - |
    if ! [[ -x "$(command -v packer)" ]]; then
      curl -skL -O https://releases.hashicorp.com/packer/1.7.2/packer_1.7.2_linux_amd64.zip
      sudo unzip -qq -o -d /usr/local/bin packer_1.7.2_linux_amd64.zip
      rm -rf packer_1.7.2_linux_amd64.zip
      curl -skL -O https://github.com/alvistack/packer-plugin-docker/releases/download/v0.0.7/packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
      sudo unzip -qq -o -d /usr/local/bin packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
      rm -rf packer-plugin-docker_v0.0.7_x5.0_linux_amd64.zip
    fi

In case single unified binary filename used, we could simply unzip it into /usr/local/bin directly without additional rename ;-)

Potential configuration

Potential References

log output of docker communicator commands

This issue was originally opened by @gfx as hashicorp/packer#4935. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

I use packer to build Docker images. packer build ... on local Mac succeeds but it fails on Circle CI 2.0, even there're some cases that it fails with cirlceci CLI.

Unfortunately, the error logs have no information about it. How can I debug it?

packer version: 1.0.0
Host platform: Docker (image base: amazonlinux:latest)
logs:

2017/05/26 05:35:52 ui:     docker: Container ID: 309cb52e190e2d07d6d5a6ed5f793d477ee8555c6473cb64fefa80ead705667c
    docker: Container ID: 309cb52e190e2d07d6d5a6ed5f793d477ee8555c6473cb64fefa80ead705667c
2017/05/26 05:35:53 packer: 2017/05/26 05:35:53 Running the provision hook
2017/05/26 05:35:53 ui: ==> docker: Uploading foo.txt => /
==> docker: Uploading foo.txt => /
2017/05/26 05:35:53 packer: 2017/05/26 05:35:53 [INFO] 0 bytes written for 'uploadData'
2017/05/26 05:35:53 [INFO] 0 bytes written for 'uploadData'
2017/05/26 05:35:53 packer: 2017/05/26 05:35:53 Executing docker exec -i 309cb52e190e2d07d6d5a6ed5f793d477ee8555c6473cb64fefa80ead705667c /bin/sh -c (command cp /packer-files/upload756830470 /foo.txt):
2017/05/26 05:35:53 ui error: ==> docker: Upload failed: Upload failed with non-zero exit status: 1
==> docker: Upload failed: Upload failed with non-zero exit status: 1
2017/05/26 05:35:53 ui: ==> docker: Killing the container: 309cb52e190e2d07d6d5a6ed5f793d477ee8555c6473cb64fefa80ead705667c
==> docker: Killing the container: 309cb52e190e2d07d6d5a6ed5f793d477ee8555c6473cb64fefa80ead705667c
2017/05/26 05:35:55 ui error: Build 'docker' errored: Upload failed with non-zero exit status: 1
2017/05/26 05:35:55 Builds completed. Waiting on interrupt barrier...
2017/05/26 05:35:55 machine readable: error-count []string{"1"}
2017/05/26 05:35:55 ui error: 
==> Some builds didn't complete successfully and had errors:

(see https://circleci.com/gh/gfx/packer-on-docker-on-circleci2/3 for all the logs)

Reproducable example: https://github.com/gfx/packer-on-docker-on-circleci2

Building images in parallel and pushing them to Docker Hub results in premature logout

Overview of the Issue

When I'm building 3 different images in parallel and pushing them to Docker Hub, the first one to complete will do a docker logout causing errors in the other 2 pushes.

Reproduction Steps

Build 3 images in parallel
Push them to Docker Hub

Plugin and Packer version

Packer v1.8.5
Docker plugin v1.0.8

Simplified Packer Buildfile

# Common config to be shared among multiple sources
source "docker" "ubuntu" {
   …
}

build {
  …
  dynamic "source" {
    for_each = ["ubuntu:22.04", "ubuntu:20.04", "ubuntu:18.04"]
    labels   = ["docker.ubuntu"]
    content {
      name  = source.value
      image = source.value
    }
  }
  …

  post-processors {
    post-processor "docker-tag" {
      repository = format("%s-%s", var.docker_repo_base, split(":", source.name)[0])
      tags = [split(":", source.name)[1]]
    }
    post-processor "docker-push" {
      login          = true
      login_username = var.docker_username
      login_password = var.docker_password
    }
  }
}

Operating system and Environment details

Log Fragments and crash.log files

See the 2 denied: requested access to the resource is denied below:

==> ansible-molecule.docker.ubuntu:20.04: Running post-processor:  (type docker-tag)
    ansible-molecule.docker.ubuntu:20.04 (docker-tag): Tagging image: sha256:67aaf5e084e8be28acf2f7382d25cc3fb9a252cf8a4477fc84229e20805e7fd2
    ansible-molecule.docker.ubuntu:20.04 (docker-tag): Repository: huyz/ansible-molecule-ubuntu:20.04
==> ansible-molecule.docker.ubuntu:20.04: Running post-processor:  (type docker-push)
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Creating temporary Docker configuration directory
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Logging in...
    ansible-molecule.docker.ubuntu:18.04:     Running setup.py install for ansible: finished with status 'done'
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Login Succeeded
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Pushing: huyz/ansible-molecule-ubuntu:20.04
    ansible-molecule.docker.ubuntu:20.04 (docker-push): The push refers to repository [docker.io/huyz/ansible-molecule-ubuntu]
==> ansible-molecule.docker.ubuntu:18.04: Committing the container
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 56fc03fd05e4: Layer already exists
==> ansible-molecule.docker.ubuntu:18.04: Running post-processor:  (type docker-tag)
    ansible-molecule.docker.ubuntu:18.04 (docker-tag): Tagging image: sha256:2f18df89a8c6a144a353ef73149a422364abcf1b8c06cdaa494cc056f14dbf84
    ansible-molecule.docker.ubuntu:18.04 (docker-tag): Repository: huyz/ansible-molecule-ubuntu:18.04
==> ansible-molecule.docker.ubuntu:18.04: Running post-processor:  (type docker-push)
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Creating temporary Docker configuration directory
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Logging in...
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Login Succeeded
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Pushing: huyz/ansible-molecule-ubuntu:18.04
    ansible-molecule.docker.ubuntu:18.04 (docker-push): The push refers to repository [docker.io/huyz/ansible-molecule-ubuntu]
    ansible-molecule.docker.ubuntu:18.04 (docker-push): 6b47c5c1f372: Preparing
    ansible-molecule.docker.ubuntu:18.04 (docker-push): a56e9bd192dc: Preparing
    ansible-molecule.docker.ubuntu:18.04 (docker-push): a56e9bd192dc: Layer already exists
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 4ef07da5ca01: Pushed
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 20.04: digest: sha256:ed4ad017978da4af4d148a0652b2964b0dc0d8f80fb18781d1ad04b29cc5f7d3 size: 742
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Pushing: huyz/ansible-molecule-ubuntu:20.04
    ansible-molecule.docker.ubuntu:20.04 (docker-push): The push refers to repository [docker.io/huyz/ansible-molecule-ubuntu]
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 4ef07da5ca01: Preparing
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 56fc03fd05e4: Preparing
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 56fc03fd05e4: Layer already exists
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 4ef07da5ca01: Layer already exists
    ansible-molecule.docker.ubuntu:20.04 (docker-push): 20.04: digest: sha256:ed4ad017978da4af4d148a0652b2964b0dc0d8f80fb18781d1ad04b29cc5f7d3 size: 742
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Logging out...
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Removing login credentials for https://index.docker.io/v1/
    ansible-molecule.docker.ubuntu:20.04 (docker-push): Removing temporary Docker configuration directory
Build 'ansible-molecule.docker.ubuntu:20.04' finished after 5 minutes 50 seconds.
    ansible-molecule.docker.ubuntu:18.04 (docker-push): 6b47c5c1f372: Pushed
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 1cdb0b846b12: Pushed
    ansible-molecule.docker.ubuntu:18.04 (docker-push): 18.04: digest: sha256:2c5886599443533a38665f1b60097a4aca09650d3a26228fd507227ad9d7e7f4 size: 742
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Pushing: huyz/ansible-molecule-ubuntu:18.04
    ansible-molecule.docker.ubuntu:18.04 (docker-push): The push refers to repository [docker.io/huyz/ansible-molecule-ubuntu]
    ansible-molecule.docker.ubuntu:18.04 (docker-push): 6b47c5c1f372: Preparing
    ansible-molecule.docker.ubuntu:18.04 (docker-push): a56e9bd192dc: Preparing
    ansible-molecule.docker.ubuntu:18.04 (docker-push): a56e9bd192dc: Layer already exists
    ansible-molecule.docker.ubuntu:18.04 (docker-push): 6b47c5c1f372: Layer already exists
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 22.04: digest: sha256:a667935e0b50c68ee8f8ff652b30d2144ec6c0199a9294160f95557b98a67436 size: 742
    ansible-molecule.docker.ubuntu:22.04 (docker-push): Pushing: huyz/ansible-molecule-ubuntu:22.04
    ansible-molecule.docker.ubuntu:22.04 (docker-push): The push refers to repository [docker.io/huyz/ansible-molecule-ubuntu]
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 1cdb0b846b12: Preparing
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 6142b8f76de4: Preparing
    ansible-molecule.docker.ubuntu:18.04 (docker-push): errors:
    ansible-molecule.docker.ubuntu:18.04 (docker-push): denied: requested access to the resource is denied
    ansible-molecule.docker.ubuntu:18.04 (docker-push): unauthorized: authentication required
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Logging out...
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Removing login credentials for https://index.docker.io/v1/
    ansible-molecule.docker.ubuntu:18.04 (docker-push): WARNING: could not erase credentials:
    ansible-molecule.docker.ubuntu:18.04 (docker-push): https://index.docker.io/v1/: error erasing credentials - err: exit status 1, out: `The specified item could not be found in the keychain.`
    ansible-molecule.docker.ubuntu:18.04 (docker-push): Removing temporary Docker configuration directory
Build 'ansible-molecule.docker.ubuntu:18.04' errored after 6 minutes 4 seconds: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 6142b8f76de4: Layer already exists
    ansible-molecule.docker.ubuntu:22.04 (docker-push): 1cdb0b846b12: Layer already exists
    ansible-molecule.docker.ubuntu:22.04 (docker-push): errors:
    ansible-molecule.docker.ubuntu:22.04 (docker-push): denied: requested access to the resource is denied
    ansible-molecule.docker.ubuntu:22.04 (docker-push): unauthorized: authentication required
    ansible-molecule.docker.ubuntu:22.04 (docker-push): Logging out...
    ansible-molecule.docker.ubuntu:22.04 (docker-push): Removing login credentials for https://index.docker.io/v1/
    ansible-molecule.docker.ubuntu:22.04 (docker-push): WARNING: could not erase credentials:
    ansible-molecule.docker.ubuntu:22.04 (docker-push): https://index.docker.io/v1/: error erasing credentials - err: exit status 1, out: `The specified item could not be found in the keychain.`
    ansible-molecule.docker.ubuntu:22.04 (docker-push): Removing temporary Docker configuration directory
Build 'ansible-molecule.docker.ubuntu:22.04' errored after 6 minutes 7 seconds: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1

Unable to push an image to ECR

Hi Packer Docker plugin team, I observed this peculiar issue.

Overview of the Issue

Post-processor docker-push fails when it tries to push an image to the ECR. Both build and login were successful. If I try to push the docker image manually (doing docker push myself) it succeeds.

Reproduction Steps

Run packer build

Plugin and Packer version

From packer version 1.8.3 and plugin version 1.0.8

Simplified Packer Buildfile

packer {
  required_plugins {
    docker = {
      version = "1.0.8"
      source  = "github.com/hashicorp/docker"
    }
  }
}

source "docker" "jdk" {
  image    = "public.ecr.aws/docker/library/openjdk:20-slim-bullseye"
  commit   = true
  platform = "linux/amd64"

  changes = [...]
}

build {
  name = "jdk_image"
  sources = [
    "source.docker.jdk"
  ]

  provisioner "shell" {
    inline = [...]
  }

  post-processors {
    post-processor "docker-tag" {
      repository = "${var.aws_account_id}.dkr.ecr.${var.aws_region}.amazonaws.com/${var.image_name}"
      tags       = [var.image_tag]
    }

    post-processor "docker-push" {
      ecr_login    = true
      login_server = "https://${var.aws_account_id}.dkr.ecr.${var.aws_region}.amazonaws.com"
    }
  }
}

Operating system and Environment details

macOS Monterey 12.6, Darwin 21.6.0 Darwin Kernel Version 21.6.0, Docker with Colima VM

Log Fragments and crash.log files

==> jdk_image.docker.jdk: Running post-processor:  (type docker-push)
    jdk_image.docker.jdk (docker-push): Creating temporary Docker configuration directory
    jdk_image.docker.jdk (docker-push): Fetching ECR credentials...
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 Getting ECR token for account: 123456789 in eu-west-1..
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 Found region eu-west-1
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 [INFO] AWS authentication used: "SharedCredentialsProvider"
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 Successfully got login for ECR: https://123456789.dkr.ecr.eu-west-1.amazonaws.com
    jdk_image.docker.jdk (docker-push): Logging in...
2022/10/27 17:23:45 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:45 Executing: /usr/local/bin/docker [--config /var/folders/d_/37637p2n7xz6nqdk5_ks2znh0000gn/T/packer2477848013 login -u AWS --password-stdin https://123456789.dkr.ecr.eu-west-1.amazonaws.com]
    jdk_image.docker.jdk (docker-push): WARNING! Your password will be stored unencrypted in /var/folders/d_/37637p2n7xz6nqdk5_ks2znh0000gn/T/packer2477848013/config.json.
    jdk_image.docker.jdk (docker-push): Configure a credential helper to remove this warning. See
    jdk_image.docker.jdk (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    jdk_image.docker.jdk (docker-push): Login Succeeded
    jdk_image.docker.jdk (docker-push): Pushing: 123456789.dkr.ecr.eu-west-1.amazonaws.com/push-repo:latest
2022/10/27 17:23:46 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:46 Executing: /usr/local/bin/docker [--config /var/folders/d_/37637p2n7xz6nqdk5_ks2znh0000gn/T/packer2477848013 push 123456789.dkr.ecr.eu-west-1.amazonaws.com/repo:latest]
    jdk_image.docker.jdk (docker-push): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
    jdk_image.docker.jdk (docker-push): Logging out...
2022/10/27 17:23:46 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:46 Executing: /usr/local/bin/docker [--config /var/folders/d_/37637p2n7xz6nqdk5_ks2znh0000gn/T/packer2477848013 logout https://123456789.dkr.ecr.eu-west-1.amazonaws.com]
    jdk_image.docker.jdk (docker-push): Removing login credentials for 123456789.dkr.ecr.eu-west-1.amazonaws.com
    jdk_image.docker.jdk (docker-push): Removing temporary Docker configuration directory
2022/10/27 17:23:46 packer-plugin-docker_v1.0.8_x5.0_darwin_amd64 plugin: 2022/10/27 17:23:46 error: Bad exit status: 1
2022/10/27 17:23:46 [INFO] (telemetry) ending docker-push

* Post-processor failed: Bad exit status: 1

Workaround for #8120: "Unable to use packer with distroless base images"

This issue was originally opened by @mprzybylski in hashicorp/packer#11283 and has been migrated to this repository. The original issue description is below.

Hello,

I took @SwampDragons up on her suggestion in issue #8120 to "tinker and can figure out how to make (distroless images) work":

Start from one of the distroless "debug" images. These are identical to the base images except that they have busybox installed.
Change the run_command for the docker source to ["-d", "-i", "-t", "--entrypoint=/busybox/sh", "--", "{{.Image}}"]

Create a symlink from /busybox/sh to /bin/sh to make shell provisioners work:

provisioner "shell-local"{
  inline = ["docker exec ${build.ID} ln -s /busybox/sh /bin/sh"]
}

Optional: run a shell-local, docker-exec-hack provisioner to delete the /busybox directory after all other provisioning is complete:
```
provisioner "shell-local" {
  only   = ["docker.ebpf_sensor"]
  inline = ["docker exec ${build.ID} rm -rf /bin/sh /busybox"]
}
```

Hope this helps!

-Mike Przybylski

file provisioner changes permissions of parent directory when uploading directory

This issue was originally opened by @GeorgFleig as hashicorp/packer#5955. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Packer version from packer version
v1.2.1
Host platform
Debian Buster
Debug log output from PACKER_LOG=1 packer build template.json.
https://gist.github.com/GeorgFleig/39a7e6703429486f41408c8ac677cc50#file-debug_log-txt
The simplest example template and scripts needed to reproduce the bug.
https://gist.github.com/GeorgFleig/39a7e6703429486f41408c8ac677cc50#file-test-json

When I use the file provisioner to upload files into a Docker container in a folder not owned by root, the permissions of the parent folder of the destination are changed.

Example:
before copy

    docker: drwxr-xr-x 2 build build 4096 Feb 28 14:57 .
    docker: drwxr-xr-x 1 root  root  4096 Feb 28 14:57 ..
    docker: -rw-r--r-- 1 build build  220 May 15  2017 .bash_logout
    docker: -rw-r--r-- 1 build build 3526 May 15  2017 .bashrc
    docker: -rw-r--r-- 1 build build  675 May 15  2017 .profile

after copy

    docker: drwxr-xr-x 2 root build 4096 Feb 28 14:57 .
    docker: drwxr-xr-x 1 root root  4096 Feb 28 14:57 ..
    docker: -rw-r--r-- 1 root build  220 May 15  2017 .bash_logout
    docker: -rw-r--r-- 1 root build 3526 May 15  2017 .bashrc
    docker: -rw-r--r-- 1 root build  675 May 15  2017 .profile
    docker: -rw-r--r-- 1 root build  538 Feb 28 10:20 file1.json
    docker: -rwxr-xr-x 1 root build 2704 Feb 28 12:23 file1.json

The owner of the parent folder /home/build is changed from build to root.

This happens when I copy a folder or the contents of a folder into that target. When I copy only a single file, the new file is owned by root:root as expected while all other permissions stay untouched.

Shell provisioner with docker builder sets wrong UID:GID when run as non-root user - permission denied on execution

This issue was originally opened by @rbellamy as hashicorp/packer#9230. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Overview of the Issue

Shell provisioner with docker builder sets wrong UID:GID for script.

Packer is run by UID:GID of 1001:1001.
Packer shell provisioner script in Docker is mode 0600 and UID:GID of 1001:1001.
Packer Docker builder uses exec_command of 1000:1000.

Error is permission denied when provisioner attempts to execute script.

When the UID:GID of the user running packer is the same as that set for the exec_command there are no permission denied errors.

Container tmp permisions

runner@ip-10-1-1-28:~/work/github-runner/github-runner/docker-1804$ docker exec -it trusting_chaum /bin/bash
root@166711faee62:/# ls -alh /tmp/
total 36K
drwxrwxrwt 8 root root 4.0K May 14 17:05 .
drwxr-xr-x 1 root root 4.0K May 14 17:05 ..
drwxrwxrwt 2 root root 4.0K May 14 17:04 .ICE-unix
drwxrwxrwt 2 root root 4.0K May 14 17:04 .Test-unix
drwxrwxrwt 2 root root 4.0K May 14 17:04 .X11-unix
drwxrwxrwt 2 root root 4.0K May 14 17:04 .XIM-unix
drwxrwxrwt 2 root root 4.0K May 14 17:04 .font-unix
-rw------- 1 1001 1001  333 May 14 17:05 script_9721.sh
drwx------ 3 root root 4.0K May 14 17:04 systemd-private-c9c189ce8e5e439e994028992152f7ad-systemd-resolved.service-vn7nkf

Container exec user UID:GID

root@166711faee62:/# id runner
uid=1000(runner) gid=1000(runner) groups=1000(runner),27(sudo)

Packer build user UID:GID

runner@ip-10-1-1-28:~/work/github-runner/github-runner/docker-1804$ id
uid=1001(runner) gid=1001(runner) groups=1001(runner),27(sudo),999(docker)

Reproduction Steps

Run packer as UID:GID of 1001:1001.
Set exec_command to UID:GID of 1000:1000.

Packer version

Packer version: 1.5.6 [go1.14.2 linux amd64]

Simplified Packer Buildfile

https://github.com/terradatum/github-runner/blob/master/docker-ubuntu1804.json

Operating system and Environment details

Host OS: Ubuntu 18.04
Docker version 19.03.8, build afacb8b7f0
Docker container OS: Ubuntu 18.04

Log Fragments and crash.log files

https://github.com/terradatum/github-runner/runs/675100587

Release 1.0.2 docker-push/tag post processors throwing errors

Overview of the Issue

After the update from 1.0.1 to 1.0.2 running packer templates which use the docker-push/tag post-processors throw the following error:

Error determining pushed Docker image digest

This seems to be generated by newly added code in the push/tag processors. Although the same/similar code used to be in the pull step code.

Also note that in the pull step it notifies the user that the digest information may not be available until the image has been pushed.

Final note: Although an error is generated the build does not fail and the end result is as expected.

Reproduction Steps

Simply run packer against the tutorial code at https://learn.hashicorp.com/tutorials/packer/docker-get-started-post-processors?in=packer/docker-get-started

Plugin and Packer version

v1.7.5

Simplified Packer Buildfile

https://gist.github.com/jjs105/751935fe509c2026ac943d4420e39e62

Operating system and Environment details

Running on Alpine via Docker

Log Fragments and crash.log files

https://gist.github.com/jjs105/d63efd5b14c53377668d0b25fa48e681

Parallel Docker pushes still failing due to credentials being removed

When filing a bug, please include the following headings if possible. Any
example text in this template can be deleted.

Overview of the Issue

#96 added support for parallel pushing images in their own context. But I'm still having an issue where authentication fails for parallel builds when the first build removes the authentication and the second fails if it is still in the middle of pushing images.

Reproduction Steps

Steps to reproduce this issue

Build multiple containers and have the post processor try to push them.

Plugin and Packer version

❯ packer version
Packer v1.8.2

❯ packer plugins installed
/usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.0.4_x5.0_darwin_amd64
/usr/local/bin/github.com/hashicorp/docker/packer-plugin-docker_v1.0.5_x5.0_darwin_amd64
/usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.1.0_x5.0_darwin_amd64

Simplified Packer Buildfile

packer {
  required_plugins {
    docker = {
      version = "1.0.5"
      source  = "github.com/hashicorp/docker"
    }
  }
}

variable "docker_password" {
  type        = string
  description = "Docker password"
  sensitive   = true
}

variable "docker_username" {
  type        = string
  description = "Docker username"
  default     = "andyshinn"
}

source "docker" "ubuntu-bionic" {
  image  = "ubuntu:18.04"
  commit = true
}

source "docker" "ubuntu-focal" {
  image  = "ubuntu:20.04"
  commit = true
}

build {
  name = "base-fips"

  sources = [
    "source.docker.ubuntu-bionic",
    "source.docker.ubuntu-focal",
  ]

  provisioner "shell" {
    only = ["docker.ubuntu-bionic", "docker.ubuntu-focal"]

    inline = [
      "apt-get update -q",
      "apt-get install -qy sudo lsb-release",
    ]
  }

  post-processors {
    post-processor "docker-tag" {
      only = ["docker.ubuntu-bionic", "docker.ubuntu-focal"]

      repository = "ghcr.io/myorg/${source.name}"
      tags       = ["0.0.3-fips"]
    }

    post-processor "docker-push" {
      only = ["docker.ubuntu-bionic", "docker.ubuntu-focal"]

      login          = true
      login_server   = "ghcr.io"
      login_username = var.docker_username
      login_password = var.docker_password
    }
  }
}

Operating system and Environment details

OS, Architecture, and any other information you can provide about the
environment.

macOS 12.4 21F79 x86_6

Log Fragments and crash.log files

Include appropriate log fragments. If the log is longer than a few dozen lines,
please include the URL to the gist of the log or
use the Github detailed format instead of posting it directly in the issue.

Set the env var PACKER_LOG=1 for maximum log detail.

    base-fips.docker.ubuntu-focal (docker-push): Login Succeeded
    base-fips.docker.ubuntu-focal (docker-push): Pushing: ghcr.io/myorg/ubuntu-focal:0.0.3-fips
    base-fips.docker.ubuntu-focal (docker-push): The push refers to repository [ghcr.io/myorg/ubuntu-focal]
    base-fips.docker.ubuntu-focal (docker-push): 5b5a60a65e35: Preparing
    base-fips.docker.ubuntu-focal (docker-push): af7ed92504ae: Preparing
    base-fips.docker.ubuntu-focal (docker-push): af7ed92504ae: Layer already exists
    base-fips.docker.ubuntu-bionic (docker-push): 5bc6fff2eeb6: Pushed
    base-fips.docker.ubuntu-bionic (docker-push): 0.0.3-fips: digest: sha256:2122845b860f1f738e4ad3d293f74f08c09eba0655cff09844844ba764f19958 size: 741
    base-fips.docker.ubuntu-bionic (docker-push): Pushing: ghcr.io/myorg/ubuntu-bionic:0.0.3-fips
    base-fips.docker.ubuntu-bionic (docker-push): The push refers to repository [ghcr.io/myorg/ubuntu-bionic]
    base-fips.docker.ubuntu-bionic (docker-push): 5bc6fff2eeb6: Preparing
    base-fips.docker.ubuntu-bionic (docker-push): 95129a5fe07e: Preparing
    base-fips.docker.ubuntu-bionic (docker-push): 95129a5fe07e: Layer already exists
    base-fips.docker.ubuntu-bionic (docker-push): 5bc6fff2eeb6: Layer already exists
    base-fips.docker.ubuntu-bionic (docker-push): 0.0.3-fips: digest: sha256:2122845b860f1f738e4ad3d293f74f08c09eba0655cff09844844ba764f19958 size: 741
    base-fips.docker.ubuntu-bionic (docker-push): Logging out...
    base-fips.docker.ubuntu-bionic (docker-push): Removing login credentials for ghcr.io
    base-fips.docker.ubuntu-bionic (docker-push): Removing temporary Docker configuration directory
Build 'base-fips.docker.ubuntu-bionic' finished after 1 minute 36 seconds.
    base-fips.docker.ubuntu-focal (docker-push): 5b5a60a65e35: Pushed
    base-fips.docker.ubuntu-focal (docker-push): 0.0.3-fips: digest: sha256:571c22dba3fbf736113d8e5de9c0d92e32167fd1d70b45e8c88e4b40e4e70367 size: 742
    base-fips.docker.ubuntu-focal (docker-push): Pushing: ghcr.io/myorg/ubuntu-focal:0.0.3-fips
    base-fips.docker.ubuntu-focal (docker-push): The push refers to repository [ghcr.io/myorg/ubuntu-focal]
    base-fips.docker.ubuntu-focal (docker-push): 5b5a60a65e35: Preparing
    base-fips.docker.ubuntu-focal (docker-push): af7ed92504ae: Preparing
    base-fips.docker.ubuntu-focal (docker-push): unauthorized: unauthenticated: User cannot be authenticated with the token provided.
    base-fips.docker.ubuntu-focal (docker-push): Logging out...
    base-fips.docker.ubuntu-focal (docker-push): Removing login credentials for ghcr.io
    base-fips.docker.ubuntu-focal (docker-push): Removing temporary Docker configuration directory

/bin/sh: 1: /home/ubuntu/packer/script_2501.sh: Permission denied

This issue was originally opened by @fredleger in hashicorp/packer#11968 and has been migrated to this repository. The original issue description is below.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Overview of the Issue

As stated in #5307 packer even at latest version upload temporary script files with wrong owner if the container is nut run as root

Reproduction Steps

source "docker" "container" {
  commit = true
  image  = "ubuntu:jammy"
  run_command = [
    "-d", "-i", "-t", "--user", "1000", "--entrypoint=/bin/sh", "--", "{{.Image}}"
  ]
}

Packer version

v1.8.3

Simplified Packer Template

see above

Operating system and Environment details

macosx ruuning rancher desktop for docker (so at the end linux)

Log Fragments and crash.log files

==> docker.container: Starting docker container...
    docker.container: Run command: docker run -v /Users/frederic/.packer.d/tmp2545466742:/packer-files -d -i -t --user 1000 --entrypoint=/bin/sh -- ubuntu:jammy
    docker.container: Container ID: eee892b4cf201e56176d93c5cd131b2ed6c76f4e4d342b8eed7c7479833718cc
==> docker.container: Pausing after run of step 'StepRun'. Press enter to continue. 
==> docker.container: Using docker communicator to connect: 172.17.0.2
==> docker.container: Pausing after run of step 'StepConnect'. Press enter to continue. 
==> docker.container: Pausing before the next provisioner . Press enter to continue. 
==> docker.container: Provisioning with shell script: /var/folders/n_/jtqch7ls28534bbt4hl905mr0000gn/T/packer-shell1344318242
==> docker.container: chmod: changing permissions of '/home/ubuntu/packer/script_2501.sh': Operation not permitted
==> docker.container: /bin/sh: 1: /home/ubuntu/packer/script_2501.sh: Permission denied
==> docker.container: Script exited with non-zero exit status: 126. Allowed exit codes are: [0]
==> docker.container: Step "StepProvision" failed

In the container:

buntu@3c5cc3a18ffc:~/packer$ ls -l
total 4
-rw------- 1 501 dialout 66 Sep  1 15:02 script_4966.sh

The owner id is 501 (which is my host user id) and not 1000 as it should be

Build docker image in parallel and credentials are removed before all images are push to ECR

This issue was originally opened by @kenit in hashicorp/packer#11358 and has been migrated to this repository. The original issue description is below.

Overview of the Issue

I am trying to build two images in parallel base on two different version PHP image and to push artifacts into AWS ECR.
When one of two artifacts is pushed to ECR, Packer will remove the login credential of ECR and break the push process of another artifact.

Packer version

1.7.7

Simplified Packer Template

source "docker" "php" {
  image  = "php:${var.php_version}-fpm"
  commit = true
}

source "docker" "php-cli" {
  image  = "php:${var.php_version}-cli"
  commit = true
}

build {
  sources = [
    "source.docker.php",
    "source.docker.php-cli"
  ]

  provisioner "shell" {
    inline = [
      "apt-get update",
      "apt-get install -y python3 git"
    ]
  }

  post-processors {
    post-processor "docker-tag" {
      repository = var.repository
      tags       = ["php-fpm"]
      only       = ["docker.php"]
    }

   post-processor "docker-tag" {
     repository = var.repository
     tags       = ["php-cli"]
     only       = ["docker.php-cli"]
   }

    post-processor "docker-push" {
      ecr_login    = true
      login_server = split("/", var.repository)[0]
    }
  }

}

Operating system and Environment details

OS: Ubuntu

Log Fragments and crash.log files

==> docker.php: Running post-processor:  (type docker-push)
    docker.php (docker-push): Fetching ECR credentials...
    docker.php (docker-push): Logging in...
    docker.php (docker-push): WARNING! Your password will be stored unencrypted in /var/lib/jenkins/.docker/config.json.
    docker.php (docker-push): Configure a credential helper to remove this warning. See
    docker.php (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    docker.php (docker-push): Login Succeeded
    docker.php (docker-push): Pushing: my.registry/test:php-fpm
    docker.php (docker-push): The push refers to repository [my.registry/test]
    docker.php (docker-push): 35a62a98fd04: Preparing
    docker.php (docker-push): a4a6079a9397: Preparing
    docker.php (docker-push): 08063f6b895d: Preparing
    docker.php (docker-push): 9de2214566a7: Preparing
    docker.php (docker-push): 09e59befc5ab: Preparing
    docker.php (docker-push): 18a6936c66e8: Preparing
    docker.php (docker-push): f8514c5993f9: Preparing
    docker.php (docker-push): 42cecd8ee840: Preparing
    docker.php (docker-push): d23471ea5612: Preparing
    docker.php (docker-push): 0392b5fdaffc: Preparing
    docker.php (docker-push): 4d3bf3167875: Preparing
    docker.php (docker-push): 42cecd8ee840: Waiting
    docker.php (docker-push): d23471ea5612: Waiting
    docker.php (docker-push): 0392b5fdaffc: Waiting
    docker.php (docker-push): 4d3bf3167875: Waiting
    docker.php (docker-push): 18a6936c66e8: Waiting
    docker.php (docker-push): f8514c5993f9: Waiting
    docker.php (docker-push): a4a6079a9397: Layer already exists
    docker.php (docker-push): 08063f6b895d: Layer already exists
    docker.php (docker-push): 9de2214566a7: Layer already exists
    docker.php (docker-push): 09e59befc5ab: Layer already exists
    docker.php (docker-push): 18a6936c66e8: Layer already exists
    docker.php (docker-push): d23471ea5612: Layer already exists
    docker.php (docker-push): 42cecd8ee840: Layer already exists
    docker.php (docker-push): f8514c5993f9: Layer already exists
    docker.php (docker-push): 0392b5fdaffc: Layer already exists
    docker.php (docker-push): 4d3bf3167875: Layer already exists
    docker.php-cli: changed: [default]
    docker.php-cli:
    docker.php-cli: TASK [Remove php source code] **************************************************
    docker.php-cli: changed: [default]
    docker.php-cli:
    docker.php-cli: PLAY RECAP *********************************************************************
    docker.php-cli: default                    : ok=12   changed=8    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
    docker.php-cli:
==> docker.php-cli: Committing the container
    docker.php-cli: Image ID: sha256:7372332cddca1e524c0e52f29e5a618b287e9e7aea30af1536c17de3c1deecb7
==> docker.php-cli: Killing the container: 9bafc10fe1a2cc27bcda7e36ffc28b7d2954e09fa579258a86dc45933d7b2f45
==> docker.php-cli: Running post-processor:  (type docker-tag)
    docker.php-cli (docker-tag): Tagging image: sha256:7372332cddca1e524c0e52f29e5a618b287e9e7aea30af1536c17de3c1deecb7
    docker.php-cli (docker-tag): Repository: my.registry/test:php-cli
==> docker.php-cli: Running post-processor:  (type docker-push)
    docker.php-cli (docker-push): Fetching ECR credentials...
    docker.php-cli (docker-push): Logging in...
    docker.php-cli (docker-push): WARNING! Your password will be stored unencrypted in /var/lib/jenkins/.docker/config.json.
    docker.php-cli (docker-push): Configure a credential helper to remove this warning. See
    docker.php-cli (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    docker.php-cli (docker-push): Login Succeeded
    docker.php-cli (docker-push): Pushing: my.registry/test:php-cli
    docker.php-cli (docker-push): The push refers to repository [my.registry/test]
    docker.php-cli (docker-push): 2bbb3d367de9: Preparing
    docker.php-cli (docker-push): bf16543ffe11: Preparing
    docker.php-cli (docker-push): b4b4b803baee: Preparing
    docker.php-cli (docker-push): 3a9949a759cd: Preparing
    docker.php-cli (docker-push): b601465815f8: Preparing
    docker.php-cli (docker-push): 3aa87050994e: Preparing
    docker.php-cli (docker-push): 42cecd8ee840: Preparing
    docker.php-cli (docker-push): d23471ea5612: Preparing
    docker.php-cli (docker-push): 0392b5fdaffc: Preparing
    docker.php-cli (docker-push): 4d3bf3167875: Preparing
    docker.php-cli (docker-push): b601465815f8: Waiting
    docker.php-cli (docker-push): 3aa87050994e: Waiting
    docker.php-cli (docker-push): 42cecd8ee840: Waiting
    docker.php-cli (docker-push): d23471ea5612: Waiting
    docker.php-cli (docker-push): 0392b5fdaffc: Waiting
    docker.php-cli (docker-push): 4d3bf3167875: Waiting
    docker.php-cli (docker-push): 3a9949a759cd: Layer already exists
    docker.php-cli (docker-push): b4b4b803baee: Layer already exists
    docker.php-cli (docker-push): bf16543ffe11: Layer already exists
    docker.php-cli (docker-push): b601465815f8: Layer already exists
    docker.php-cli (docker-push): 42cecd8ee840: Layer already exists
    docker.php-cli (docker-push): 3aa87050994e: Layer already exists
    docker.php-cli (docker-push): d23471ea5612: Layer already exists
    docker.php-cli (docker-push): 0392b5fdaffc: Layer already exists
    docker.php-cli (docker-push): 4d3bf3167875: Layer already exists
    docker.php (docker-push): 35a62a98fd04: Pushed
    docker.php (docker-push): php-fpm: digest: sha256:2d39cd4bd30195d527eef4189bf11a6c38dd246d67df9b087dd4ab2762f78a63 size: 2622
    docker.php (docker-push): Pushing: my.registry/test:php-fpm
    docker.php (docker-push): The push refers to repository [my.registry/test]
    docker.php (docker-push): 35a62a98fd04: Preparing
    docker.php (docker-push): a4a6079a9397: Preparing
    docker.php (docker-push): 08063f6b895d: Preparing
    docker.php (docker-push): 9de2214566a7: Preparing
    docker.php (docker-push): 09e59befc5ab: Preparing
    docker.php (docker-push): 18a6936c66e8: Preparing
    docker.php (docker-push): f8514c5993f9: Preparing
    docker.php (docker-push): 42cecd8ee840: Preparing
    docker.php (docker-push): d23471ea5612: Preparing
    docker.php (docker-push): 0392b5fdaffc: Preparing
    docker.php (docker-push): 4d3bf3167875: Preparing
    docker.php (docker-push): 09e59befc5ab: Waiting
    docker.php (docker-push): 18a6936c66e8: Waiting
    docker.php (docker-push): f8514c5993f9: Waiting
    docker.php (docker-push): 42cecd8ee840: Waiting
    docker.php (docker-push): d23471ea5612: Waiting
    docker.php (docker-push): 0392b5fdaffc: Waiting
    docker.php (docker-push): 4d3bf3167875: Waiting
    docker.php (docker-push): 08063f6b895d: Layer already exists
    docker.php (docker-push): 9de2214566a7: Layer already exists
    docker.php (docker-push): a4a6079a9397: Layer already exists
    docker.php (docker-push): 35a62a98fd04: Layer already exists
    docker.php (docker-push): 09e59befc5ab: Layer already exists
    docker.php (docker-push): 18a6936c66e8: Layer already exists
    docker.php (docker-push): 42cecd8ee840: Layer already exists
    docker.php (docker-push): f8514c5993f9: Layer already exists
    docker.php (docker-push): d23471ea5612: Layer already exists
    docker.php (docker-push): 0392b5fdaffc: Layer already exists
    docker.php (docker-push): 4d3bf3167875: Layer already exists
    docker.php (docker-push): php-fpm: digest: sha256:2d39cd4bd30195d527eef4189bf11a6c38dd246d67df9b087dd4ab2762f78a63 size: 2622
    docker.php (docker-push): Logging out...
    docker.php (docker-push): Removing login credentials for my.registry
Build 'docker.php' finished after 3 minutes 24 seconds.
    docker.php-cli (docker-push): 2bbb3d367de9: Pushed
    docker.php-cli (docker-push): php-cli: digest: sha256:d261fc7ad33d6417ddc4ce4584dc7506c19a52dc908349f080f246217104f222 size: 2413
    docker.php-cli (docker-push): Pushing: my.registry/test:php-cli
    docker.php-cli (docker-push): The push refers to repository [my.registry/test]
    docker.php-cli (docker-push): 2bbb3d367de9: Preparing
    docker.php-cli (docker-push): bf16543ffe11: Preparing
    docker.php-cli (docker-push): b4b4b803baee: Preparing
    docker.php-cli (docker-push): 3a9949a759cd: Preparing
    docker.php-cli (docker-push): b601465815f8: Preparing
    docker.php-cli (docker-push): 3aa87050994e: Preparing
    docker.php-cli (docker-push): 42cecd8ee840: Preparing
    docker.php-cli (docker-push): d23471ea5612: Preparing
    docker.php-cli (docker-push): 0392b5fdaffc: Preparing
    docker.php-cli (docker-push): 4d3bf3167875: Preparing
    docker.php-cli (docker-push): no basic auth credentials
    docker.php-cli (docker-push): Logging out...
    docker.php-cli (docker-push): Removing login credentials for my.registry
Build 'docker.php-cli' errored after 3 minutes 33 seconds: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1

==> Wait completed after 3 minutes 33 seconds

==> Some builds didn't complete successfully and had errors:
--> docker.php-cli: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1

==> Builds finished. The artifacts of successful builds are:
--> docker.php: Imported Docker image: sha256:97d8a3f4708ccde2c9dce2a7988f3dd6df5462dc1deb42c464a67f23eca90854
--> docker.php: Imported Docker image: my.registry/test:php-fpm with tags my.registry/test:php-fpm

Docker builder hangs when using Ansible provisioner in Docker in Docker

This issue was originally opened by @frodopwns as hashicorp/packer#3057.It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

I am running Jenkins in a Ubuntu 14.04 Docker Container.
Tried with docker 1.7.1 and 1.9.1.

The jenkins image has these volumes mounted in to allow for docker-in-docker:

-v /usr/bin/docker:/usr/bin/docker
-v /var/run/docker.sock:/var/run/docker.sock

Inside this container I run a packer build with a docker builder and an ansible provisioner. This build works fine when I run it on a normal host running docker.

When run inside a docker-based jenkins image it hangs at:

==> docker: Starting docker container...
    docker: Run command: docker run -v /root/.ssh:/root/.ssh -v /etc/ansible:/etc/ansible -v /tmp/packer-docker574433279:/packer-files -d -i -t some-user/some-image /bin/bash
    docker: Container ID: 12712e6e1d7a78fef387130ff5f1f849f98333ce6eeac31e292725c5006b6355
==> docker: Provisioning with Ansible...
    docker: Creating Ansible staging directory...
    docker: Creating directory: /tmp/packer-provisioner-ansible-local

This will hang forever.

Is there a known issue with running the docker builder with ansible provisioner in a docker-in-docker environment? If so what can we do about it? Jenkins is being deployed via docker more and more often so things like packer which often get triggered by jenkins will need to run in containers.

docker-push doesn't push to Private repository.

Overview of the Issue

When using the post-processor "docker-push" post-processor with Private repository location, Username & password settings, the push command appears to try to push the image to Dockerhub instead.

Reproduction Steps

Run a packer build using the Docker source + tag & push post-processors

build  {
  ... 
  # sources & provisioning steps
  ...
  post-processor "docker-tag" {
    repository = "${var.container_registry}/my-base-image"
    tags       = [local.container_tag]
  }

  post-processor "docker-push" {
    login              = true
    login_server   = var.container_registry
    login_username = var.container_registry_username
    login_password = var.container_registry_password
  }
}

Plugin and Packer version

1.8.6_windows_amd64
packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe

Simplified Packer Buildfile

variable "container_registry" { }
variable "container_registry_username" {}
variable "container_registry_password" {
sensitive = true
}

locals {
container_tag = "0.1.0"
}
build  {
  ... 
  # sources & provisioning steps
  ...
  post-processor "docker-tag" {
    repository = "${var.container_registry}/my-base-image"
    tags       = [local.container_tag]
  }

  post-processor "docker-push" {
    login              = true
    login_server   = var.container_registry
    login_username = var.container_registry_username
    login_password = var.container_registry_password
  }
}

Operating system and Environment details

Windows Server 2022 x64

Log Fragments and crash.log files

==> BaseWindowContainer.docker.winservercore: Running post-processor: (type docker-tag)

[INFO] (telemetry) Starting post-processor docker-tag
BaseWindowContainer.docker.winservercore (docker-tag): Tagging image: sha256:ddbc57331955ad9b07dc8708ad24eb51f1fadb8b3fde730f7c7afa9ef1a355b4
BaseWindowContainer.docker.winservercore (docker-tag): Repository: my-container.registery.com/my-base-image:0.1.0

==> BaseWindowContainer.docker.winservercore: Running post-processor: (type docker-push)

22-Apr-2023 12:53:27 | 2023/04/22 16:53:27 Flagging to keep original artifact from post-processor 'docker-tag'
22-Apr-2023 12:53:27 | 2023/04/22 16:53:27 [INFO] (telemetry) Starting post-processor docker-push
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): Creating temporary Docker configuration directory
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): Logging in...
22-Apr-2023 12:53:27 | 2023/04/22 16:53:27 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/04/22 16:53:27 Executing: C:\Program Files\Docker\docker.exe [--config C:\Users\User1\AppData\Local\Temp\packer2090728066 login -u reg_user--password-stdin my-container.registery.com]
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): WARNING! Your password will be stored unencrypted in C:\Users\User1\AppData\Local\Temp\packer2090728066\config.json.
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): Configure a credential helper to remove this warning. See
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): Login Succeeded
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
22-Apr-2023 12:53:27 | BaseWindowContainer.docker.winservercore (docker-push): Pushing: sha256:ddbc57331955ad9b07dc8708ad24eb51f1fadb8b3fde730f7c7afa9ef1a355b4
22-Apr-2023 12:53:28 | 2023/04/22 16:53:28 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/04/22 16:53:28 Executing: C:\Program Files\Docker\docker.exe [--config C:\Users\User1\AppData\Local\Temp\packer2090728066 push sha256:ddbc57331955ad9b07dc8708ad24eb51f1fadb8b3fde730f7c7afa9ef1a355b4]
22-Apr-2023 12:53:28 | BaseWindowContainer.docker.winservercore (docker-push): An image does not exist locally with the tag: sha256
22-Apr-2023 12:53:28 | BaseWindowContainer.docker.winservercore (docker-push): The push refers to repository [docker.io/library/sha256]
22-Apr-2023 12:53:28 | BaseWindowContainer.docker.winservercore (docker-push): Logging out...

simple docker build changing CMD and ENTRYPOINT of resulting image

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Overview of the Issue

the resulting image packer produces is changing the CMD and Entrypoint (possibly other things)

build {
  name = "amzn2-ctm-core"

  source "source.docker.dockerhub" {
    image  = "amazonlinux:2"
  }

  post-processors {
    post-processor "docker-tag" {
      tags = [
        "core_${local.timestamp}",
        "core_latest"
      ]
    }
  }
}

After building this, comparing the images:

# The source image:
$ docker inspect amazonlinux:2 | jq -s ".[0][0].Config.Cmd"
[
  "/bin/bash"
]
$ docker inspect amazonlinux:2 | jq -s ".[0][0].Config.Entrypoint"
null

# the resulting image
$ docker inspect acct1234.dkr.ecr.us-east-1.amazonaws.com/amzn2-ctm:core_20221118194409 | jq -s ".[0][0].Config.Cmd"
null
$ docker inspect acct1234.dkr.ecr.us-east-1.amazonaws.com/amzn2-ctm:core_20221118194409 | jq -s ".[0][0].Config.Entrypoint"
[
  "/bin/sh"
]

Why is packer doing this?

Packer version

1.8.4

Operating system and Environment details

built on, and sourcing from:
amazonlinux;2

error with provisioning scripts in docker

packer version 1.0.9.

similar problem in such tickets: 7578, 2697

I'm going with the tutorial, theme "Add provisioner to template"

my docker-ubuntu.pkr.hcl is:

packer {
  required_plugins {
    docker = {
      version = ">= 1.0.8"
      source = "github.com/hashicorp/docker"
    }
  }
}

source "docker" "ubuntu" {
  image  = "ubuntu:jammy"
  commit = true
}

build {
  name    = "learn-packer"
  sources = [
    "source.docker.ubuntu"
  ]
  provisioner "shell" {
    environment_vars = [
      "FOO=hello world",
    ]
    inline = [
      "echo Adding file to Docker Container",
      "echo \"FOO is $FOO\" > example.txt",
    ]
  }
}

everything ok on the build stage:

❯ packer build docker-ubuntu.pkr.hcl
learn-packer.docker.ubuntu: output will be in this color.

==> learn-packer.docker.ubuntu: Creating a temporary directory for sharing data...
==> learn-packer.docker.ubuntu: Pulling Docker image: ubuntu:jammy
    learn-packer.docker.ubuntu: jammy: Pulling from library/ubuntu
    learn-packer.docker.ubuntu: Digest: sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2
    learn-packer.docker.ubuntu: Status: Image is up to date for ubuntu:jammy
    learn-packer.docker.ubuntu: docker.io/library/ubuntu:jammy
==> learn-packer.docker.ubuntu: Starting docker container...
    learn-packer.docker.ubuntu: Run command: docker run -v /home/kay/.config/packer/tmp4060350213:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:jammy
    learn-packer.docker.ubuntu: Container ID: a4166ea59cfc8501a4f2b0dde53a90d02dba8e7e650c0aa62a06bd01385d7c40
==> learn-packer.docker.ubuntu: Using docker communicator to connect: 172.17.0.2
==> learn-packer.docker.ubuntu: Provisioning with shell script: /tmp/packer-shell3856444975
    learn-packer.docker.ubuntu: Adding file to Docker Container
==> learn-packer.docker.ubuntu: Committing the container
    learn-packer.docker.ubuntu: Image ID: sha256:d2ec7a28d25a7474f7304bb1abe4474d8cb1d4906119a32e416cecc67cf1aa81
==> learn-packer.docker.ubuntu: Killing the container: a4166ea59cfc8501a4f2b0dde53a90d02dba8e7e650c0aa62a06bd01385d7c40
Build 'learn-packer.docker.ubuntu' finished after 2 seconds 963 milliseconds.

==> Wait completed after 2 seconds 963 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> learn-packer.docker.ubuntu: Imported Docker image: sha256:d2ec7a28d25a7474f7304bb1abe4474d8cb1d4906119a32e416cecc67cf1aa81

but after it failes to run:

❯ docker run -it d2ec7a28d25a
/bin/bash: 1: Syntax error: "(" unexpected

in 1.0.8 everything is ok

Add multi-stage builds for Docker

This issue was originally opened by @finferflu as hashicorp/packer#9462. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Please search the existing issues for relevant feature requests, and use the
reaction feature
(https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/)
to add upvotes to pre-existing requests.

Feature Description

Add the ability to create multi-stage builds.

Use Case(s)

This is vital to keep the image sizes down, e.g. I have a Dockerfile which I can't translate to Packer:

FROM prom/prometheus:v2.19.0

FROM alpine:3.10.2
RUN apk add gettext

COPY --from=0 /bin/prometheus /bin/prometheus

RUN mkdir -p /prometheus /etc/prometheus && \
chown -R nobody:nogroup etc/prometheus /prometheus
# Run envsubst before Prometheus.
RUN echo $'#!/bin/sh\n\
envsubst < /etc/prometheus/orig.yml > /etc/prometheus/prometheus.yml && \
exec /bin/prometheus "$@"' \
> /etc/prometheus/entrypoint.sh
RUN chmod +x /etc/prometheus/entrypoint.sh
ENTRYPOINT ["/etc/prometheus/entrypoint.sh"]

CMD [ "--config.file=/etc/prometheus/prometheus.yml", \
"--storage.tsdb.path=/prometheus" ]
USER nobody
EXPOSE 9090
VOLUME [ "/prometheus" ]
WORKDIR /prometheus

# This is your local prometheus.yml.
ADD prometheus.yml /etc/prometheus/orig.yml

PowerShell at Windows container ignores errors

Hi Packer team,

Overview of the Issue

PowerShell provisioner ignores some type of errors although $ErrorActionPreference = 'Stop' is set and there is [INFO] 259 bytes written for 'stderr' log when docker builder and windows container is used. There is a related issue on PowerShell provisioner hashicorp/packer#4916, however this issue can be solved by putting $ErrorActionPreference = 'Stop' to the top of the scripts(like in the simplified packer buildfile) when googlecompute builder is used but same solution doesn't work for docker builder.

Helpful(?) notes

When I tried to replicate what docker provisioner does by running same command(docker exec -i ${container.id} powershell (${command})) from my local, it shows the same output and sets $LastExitCode to 1 on my local. So, I thought there could be some issues when checking errors after running commands on powershell.

Reproduction Steps

Run packer build.

Plugin and Packer version

packer: 1.8.4
plugin: 1.0.8

Simplified Packer Buildfile

packer {
  required_plugins {
    docker = {
      version = "1.0.8"
      source  = "github.com/hashicorp/docker"
    }
  }
}

source "docker" "windows" {
  image = "docker.io/mcr.microsoft.com/windows/servercore:ltsc2022"
  windows_container = true
  discard = true
}

build {
  name = "windows_image"
  sources = ["source.docker.windows"]

  provisioner "powershell" {
    inline = [
        "$ErrorActionPreference = 'Stop'",
        "echo 'Will throw an error'",
        "throw 'cmdfail'",
    ]
  }
}

Operating system and Environment details

Windows 11 Laptop, Docker version 20.10.22, build 3a2c30b

Log Fragments and crash.log files

==> windows_image.docker.windows: Using dockerWindowsContainer communicator to connect:
2023/01/25 15:34:10 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:10 Running the provision hook
2023/01/25 15:34:10 [INFO] (telemetry) Starting provisioner powershell
==> windows_image.docker.windows: Provisioning with Powershell...
2023/01/25 15:34:10 packer.exe plugin: Found command: $ErrorActionPreference = 'Stop'
2023/01/25 15:34:10 packer.exe plugin: Found command: echo 'Will throw an error'
2023/01/25 15:34:10 packer.exe plugin: Found command: throw 'cmdfail'
==> windows_image.docker.windows: Provisioning with powershell script: C:\Users\NAZIRY~1\AppData\Local\Temp\powershell-provisioner133813514
2023/01/25 15:34:10 packer.exe plugin: Opening C:\Users\NAZIRY~1\AppData\Local\Temp\powershell-provisioner133813514 for reading
2023/01/25 15:34:10 packer.exe plugin: Uploading env vars to c:/Windows/Temp/packer-ps-env-vars-63d121c2-c933-ee29-115f-58428ec45e10.ps1
2023/01/25 15:34:10 packer.exe plugin: [INFO] 69 bytes written for 'uploadData'
2023/01/25 15:34:10 [INFO] 69 bytes written for 'uploadData'
2023/01/25 15:34:10 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:10 Executing docker exec -i 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462 powershell (Copy-Item -Path c:/packer-files/upload2656082275 -Destination c:/Windows/Temp/packer-ps-env-vars-63d121c2-c933-ee29-115f-58428ec45e10.ps1):
2023/01/25 15:34:12 packer.exe plugin: [INFO] 75 bytes written for 'uploadData'
2023/01/25 15:34:12 [INFO] 75 bytes written for 'uploadData'
2023/01/25 15:34:12 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:12 Executing docker exec -i 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462 powershell (Copy-Item -Path c:/packer-files/upload4134780674 -Destination c:/Windows/Temp/script-63d121c2-9cd1-0134-9dcc-b34b157990b4.ps1):
2023/01/25 15:34:13 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:13 Executing docker exec -i 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462 powershell (powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-63d121c2-c933-ee29-115f-58428ec45e10.ps1; &'c:/Windows/Temp/script-63d121c2-9cd1-0134-9dcc-b34b157990b4.ps1'; exit $LastExitCode }"):
==> windows_image.docker.windows: cmdfail
==> windows_image.docker.windows: At C:\Windows\Temp\script-63d121c2-9cd1-0134-9dcc-b34b157990b4.ps1:3 char:1
==> windows_image.docker.windows: + throw 'cmdfail'
==> windows_image.docker.windows: + ~~~~~~~~~~~~~~~
==> windows_image.docker.windows:     + CategoryInfo          : OperationStopped: (cmdfail:String) [], RuntimeEx
==> windows_image.docker.windows:    ception
==> windows_image.docker.windows:     + FullyQualifiedErrorId : cmdfail
==> windows_image.docker.windows:
    windows_image.docker.windows: Will throw an error
2023/01/25 15:34:16 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:16 [INFO] RPC endpoint: Communicator ended with: 0
2023/01/25 15:34:16 [INFO] 259 bytes written for 'stderr'
2023/01/25 15:34:16 [INFO] 21 bytes written for 'stdout'
2023/01/25 15:34:16 [INFO] RPC client: Communicator ended with: 0
2023/01/25 15:34:16 [INFO] RPC endpoint: Communicator ended with: 0
2023/01/25 15:34:16 packer.exe plugin: [INFO] 21 bytes written for 'stdout'
2023/01/25 15:34:16 packer.exe plugin: [INFO] 259 bytes written for 'stderr'
2023/01/25 15:34:16 packer.exe plugin: [INFO] RPC client: Communicator ended with: 0
2023/01/25 15:34:16 packer.exe plugin: c:/Windows/Temp/script-63d121c2-9cd1-0134-9dcc-b34b157990b4.ps1 returned with exit code 0
2023/01/25 15:34:16 packer.exe plugin: [INFO] 511 bytes written for 'uploadData'
2023/01/25 15:34:16 [INFO] 511 bytes written for 'uploadData'
2023/01/25 15:34:16 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:16 Executing docker exec -i 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462 powershell (Copy-Item -Path c:/packer-files/upload1183844614 -Destination c:/Windows/Temp/packer-cleanup-63d121c2-4108-6f8d-01f7-cf6ac4f19e82.ps1):
2023/01/25 15:34:17 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:17 Executing docker exec -i 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462 powershell (powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){set-variable -name variable:global:ProgressPreference -value 'SilentlyContinue'};. c:/Windows/Temp/packer-ps-env-vars-63d121c2-c933-ee29-115f-58428ec45e10.ps1; &'c:/Windows/Temp/packer-cleanup-63d121c2-4108-6f8d-01f7-cf6ac4f19e82.ps1'; exit $LastExitCode }"):
2023/01/25 15:34:19 packer-plugin-docker_v1.0.8_x5.0_windows_amd64.exe plugin: 2023/01/25 15:34:19 [INFO] RPC endpoint: Communicator ended with: 0
2023/01/25 15:34:19 [INFO] 0 bytes written for 'stderr'
2023/01/25 15:34:19 [INFO] 0 bytes written for 'stdout'
2023/01/25 15:34:19 [INFO] RPC client: Communicator ended with: 0
2023/01/25 15:34:19 [INFO] RPC endpoint: Communicator ended with: 0
2023/01/25 15:34:19 packer.exe plugin: [INFO] 0 bytes written for 'stdout'
2023/01/25 15:34:19 packer.exe plugin: [INFO] 0 bytes written for 'stderr'
2023/01/25 15:34:19 packer.exe plugin: [INFO] RPC client: Communicator ended with: 0
2023/01/25 15:34:19 [INFO] (telemetry) ending powershell
==> windows_image.docker.windows: Killing the container: 9cc26487e49e0bbcc84f47406a4dbca0ee2268a0477e7362bd12d7172bd7b462
2023/01/25 15:34:19 [INFO] (telemetry) ending docker.windows
Build 'windows_image.docker.windows' finished after 15 seconds 471 milliseconds.
==> Wait completed after 15 seconds 471 milliseconds

==> Builds finished. The artifacts of successful builds are:
==> Wait completed after 15 seconds 471 milliseconds

HCL2 example of "windows_container": true uses ubuntu image

On https://developer.hashicorp.com/packer/plugins/builders/docker#docker-for-windows
this example does not use correct, does it?

File provisioner fails in 1.7.0

Hey there,

I was using 1.6.5 before and now just switched to 1.7.0 to test the new init and required_plugins features, thus I modified my template to look like this:

packer {
  required_version = ">= 1.7"

  required_plugins {

    docker = {
      version = "0.0.6"
      source  = "github.com/hashicorp/docker"
    }
  }
}

and kept the rest of it as it was.
I did a packer init template.pkr.hcl which was successful, and then did a build.
The shell provisioner worked fine but then when it reached the file one, it errored out like this:

==> docker.scenarioo: Uploading files/entrypoint.sh => /entrypoint.sh
==> docker.scenarioo: Provisioning step had errors: Running the cleanup provisioner, if present...
==> docker.scenarioo: Killing the container: e2e06f37ce42ffc1085b26f1a25a5d32c444cffa1f1bbb27a133e0ebf8f4bc3a
    docker.scenarioo: entrypoint.sh 0 B / 500 B [-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------]   0.00%
Build 'docker.scenarioo' errored after 5 seconds 507 milliseconds: [pos 6839]: readContainerLen: Unrecognized descriptor byte: hex: d7, decimal: 215

Am I missing something, or is this a known issue? The very same config worked fine prior to this upgrade.

Failed to upload to 'xxx' in container: Error response from daemon: Error processing tar file(exit status 1): chtimes <yyy>: no such file or directory

This issue was originally opened by @brikis98 as hashicorp/packer#6103. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

I have a Packer template that I'm using to build Docker images. Here's the rough idea of what the template looks like:

{
  "builders": [{
    "name": "ubuntu-docker",
    "type": "docker",
    "image": "gruntwork/ubuntu-test:16.04",
    "commit": true,
    "run_command": ["-d", "-i", "-t", "{{.Image}}", "/sbin/init"],
    "privileged": true,
    "volumes": {
      "/sys/fs/cgroup": "/sys/fs/cgroup"
    }
  }],
  "provisioners": [{
    "type": "file",
    "source": "{{template_dir}}/../../../my-folder-name",
    "destination": "/tmp"
  },{
    "type": "shell",
    "inline": [
      "/tmp/some-script.sh"
    ]
  }],
  "post-processors": [{
    "type": "docker-tag",
    "repository": "gruntwork/my-container",
    "tag": "latest",
    "only": ["ubuntu-docker"]
  }]
}

When I run packer build locally (on my Mac), the Docker container builds just fine. However, when I run my automated tests in CircleCI, which also call packer build, I see an error like this:

Failed to upload to '/tmp' in container: Error response from daemon: Error processing tar file(exit status 1): chtimes /my-folder-name/_docs: no such file or directory

The /my-folder-name/_docs folder definitely exists. The exact file/folder it fails on seems to change from run to run.

A few key notes:

I was originally running Packer builds in parallel, so I tried setting different PACKER_TMP_DIR values for each one. However, that didn't help, and upon further testing, the error happens even if the builds run sequentially.
Packer version: v1.2.2
Platform: CircleCI (Ubuntu 14.04)

ECR Login fails in Packer 1.8.3

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

When filing a bug, please include the following headings if possible. Any
example text in this template can be deleted.

Overview of the Issue

Without any changes to the Packer HCL template, Packer builds are now failing with the error:

Error fetching ECR credentials: failed to parse the ECR URL: account-id.dkr.ecr.us-east-1.amazonaws.com
==> build.docker.shift: parse "account-id.dkr.ecr.us-east-1.amazonaws.com": invalid URI for request
==> build.docker.shift: it should be either of the form `public.ecr.aws/<registry_alias>/<registry_name>` or `<account number>.dkr.ecr.<region>.amazonaws.com`
Build 'build.docker.shift' errored after 76 milliseconds 463 microseconds: Error fetching ECR credentials: failed to parse the ECR URL: account-id.dkr.ecr.us-east-1.amazonaws.com
parse "account-id.dkr.ecr.us-east-1.amazonaws.com": invalid URI for request
it should be either of the form `public.ecr.aws/<registry_alias>/<registry_name>` or `<account number>.dkr.ecr.<region>.amazonaws.com`

(I have substituted the account id number with "account-id" string above)

Reproduction Steps

Steps to reproduce this issue

packer build with an ECR login.

Packer version

1.8.3
From packer version

Operating system and Environment details

Ubuntu 22.04, macOS 12.4

Packer overrides CMD and ENTRYPOINT already set in FROM image

This issue was originally opened by @dimisjim as hashicorp/packer#10311. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Overview of the Issue

A dev familiar with Dockerfile expects Packer to keep the ENTRYPOINT and/or CMD set in the image where the build is based from. Why would Packer reset these, if they are not overridden explicitly in the commit argument of the builder?

Reproduction Steps

Create a simple nginx image based on the official one.
if no CMD is defined in the commit argument of the builder, it defaults to /bin/sh
only when "ENTRYPOINT nginx -g 'daemon off;'" the image works as expected

Packer version

1.6.5

Simplified Packer Buildfile

this is necessary

source "docker" "nginx" {
  image   = "nginx"
  commit  = true
  changes = [
    "EXPOSE 80",
    "ENTRYPOINT nginx -g 'daemon off;'"
  ]
}

this should be enough:

source "docker" "nginx" {
  image   = "nginx"
  commit  = true
  changes = [
    "EXPOSE 80",
  ]
}

considering this exists already:

Operating system and Environment details

Ubuntu 20.10,
Docker version 19.03.13, build 4484c46

related: #13

Packer's docker plugin metadata change for Env does require equal sign and cannot extend env vars in hcl2 variant language

source "docker" "ubi9-minimal" {
  image  = "proxy-registry.example.com/library/external/registry.access.redhat.com/ubi9-minimal:latest"
  commit = true
  # https://developer.hashicorp.com/packer/plugins/builders/docker#basic-example-changes-to-metadata
  changes = [
    "ENV MOO=foo",
    "ENV PATH=${local.JDK_DIR}:$PATH"
  ]
}

hcl2

Contrary to the documentation Packer v1.8.6 requires the "=" sign to alter the environment variables
Cannot extend (inherit and add) existing env vars like PATH

Please fix the doc and add that feature if I am correct.

Thanks!

Support `--platform` docker CLI argument to target the image architecture to be produced.

Community Note

Description

Support for docker's --platform option added to docker pull, to support cross arch image builds from differing architecture host OS's

Use Case(s)

packer use aarch64 operating systems such as M1 and other M Series Apple silicon, future Thinkpad X13s snapdragon use

Potential configuration

Amendment of this section

packer-plugin-docker/builder/docker/driver_docker.go

Line 275 in 0be3f45

cmd := exec.Command("docker", "pull", image)

to permit the passing of the platform/arch target (injecting --platform e.g. docker pull --platform=linux/amd64 image:TAG (https://docs.docker.com/engine/reference/commandline/pull/#options)

Potential References

https://docs.docker.com/engine/reference/commandline/pull/#options

hcl file snippet:

...
source "docker" "oraclelinux" {
  image  = "--platform=linux/amd64 oraclelinux:8"
  commit = true
}
...

packer build -debug:

==> oraclelinux.docker.oraclelinux: Pausing after run of step 'StepTempDir'. Press enter to continue. 
==> oraclelinux.docker.oraclelinux: Pulling Docker image: --platform=linux/amd64 oraclelinux:8
    oraclelinux.docker.oraclelinux: "docker pull" requires exactly 1 argument.
    oraclelinux.docker.oraclelinux: See 'docker pull --help'.
    oraclelinux.docker.oraclelinux: Usage:  docker pull [OPTIONS] NAME[:TAG|@DIGEST]
    oraclelinux.docker.oraclelinux: Pull an image or a repository from a registry

Image string does not appear to be passed verbatim, as the following shows this is valid:

docker pull --platform=linux/amd64 oraclelinux:8
8: Pulling from library/oraclelinux
Digest: sha256:a0b7d2abffd01d284bd0db50c7f45abec5f8585e8a4cbf14a172a7a0362c2759
Status: Image is up to date for oraclelinux:8
docker.io/library/oraclelinux:8

Missing License

Overview of the Issue

This repository is missing any hints towards the applicable open source license. This makes it impossible to use packer-plugin-docker beyond viewing and forking the source code as guaranteed by the GitHub TOS.

From the looks of it, this problem may also affect Packer itself, because many versions also include source code from this repository.

If the code originates from packer, then it must also provided under the terms of the MPL 2.0 (or a compatible license), here.
If the code originates from here, then packer as a whole may not be provided under the MPL 2.0, unless the copyright holders/contributors also chose to provided it under MPL 2.0 over at packer, but not here (which the outside world has no way of knowing).

Could you please shed some light on this?

Reproduction Steps

Check for a LICENSE file in the expected places.
Find none.

Plugin and Packer version

Affects all versions/revisions of packer-plugin-docker.

Copies of packer-plugin-docker source code included in packer versions up to v1.7.2 (removed in hashicorp/packer@0f6a081).

docker builder: double backslash breaks exec form of changes

This issue was originally opened by @Yoshiiiiiii as hashicorp/packer#9092. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

The docker builder for windows images behaves unexpected when trying to change the ENTRYPOINT and CMD of the image. Double backslash breaks exec form of changes and turns it into shell form. Escaping the backslash with another backslash felt like a thing that should have worked.

Overview of the Issue

I was trying to build an image that executes a powershell script on startup. I tried to change the ENTRYPOINT and the CMD the following way:

"changes": [
    "ENTRYPOINT [\"powershell\", \"-NoExit\", \"-Command\", \"Set-Location\", \"C:/Setup\", \";\"]",
    "CMD [\".\\Start.ps1\"]"
]

This resulted in the following image:

"Cmd": [
	"cmd /S /C [\".\\Start.ps1\"]"
],
"ArgsEscaped": true,
"Image": "mcr.microsoft.com/windows/servercore:1809",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
	"powershell",
	"-NoExit",
	"-Command",
	"Set-Location",
	"C:/Setup",
	";"
],

So it turned the change of the CMD into shell form. "CMD [\"./Start.ps1\"]" does work. It took a while to figure this out. I couldn't find anything in the docs that adresses this. Maybe it was just me approaching this the wrong way.

Before doing it correct I tried this:

"changes": [
  "ENTRYPOINT [\"powershell\", \"-NoExit\", \"-Command\", \".\\Start.ps1\", \";\"]",
  "WORKDIR C:/Setup"
]

which resulted into this via docker inspect:

"WorkingDir": "C:\\Setup",
"Entrypoint": [
  "cmd /S /C [\"powershell\", \"-NoExit\", \"-Command\", \".\\Start.ps1\"]"
]

So the WORKINGDIR is translated into double backslashes from docker. And you can see that the backslashes also break the exec form of the ENTRYPOINT.

Reproduction Steps

Try to escape '\' in a change like this '\\' and the docker builder is not able to detect the exec form of the CMD. Same with the ENTRYPOINT.

Packer version

From packer_1.5.5_windows_amd64.

Simplified Packer Buildfile

{
  "builders": [
    {
      "type": "docker",
      "image": "mcr.microsoft.com/windows/servercore:1809",
      "container_dir": "c:/Temp",
      "windows_container": true,
      "commit": true,
      "changes": [
        "ENTRYPOINT [\"powershell\", \"-NoExit\", \"-Command\", \"Set-Location\", \"C:/Setup\", \";\"]",
        "CMD [\".\\Start.ps1\"]"
      ]
    }
  ],
  "provisioners": [
    {
      "type": "powershell",
      "inline": [
        "New-Item -Path 'c:\\' -Name 'Setup' -ItemType 'directory'"
      ]
    },
    {
      "type": "file",
      "source": "{{ template_dir }}/Start.ps1",
      "destination": "C:/Setup/Start.ps1"
    }
  ]  
}

Operating system and Environment details

Windows Server 2019 Datacenter

Unable to push to ECR private repo in 1.0.7

Overview of the Issue

1.0.7 introduces some new code for ECR repos but it appears to fail to parse an ECR private repo login_server correctly. Reverting to 1.0.6 fixes the issue.

Reproduction Steps

packer build

Plugin and Packer version

Packer version 1.8.0
Docker plugin 1.0.7

Simplified Packer Buildfile

post-processor "docker-push" {
      ecr_login    = true
      login_server = "317215053986.dkr.ecr.us-east-2.amazonaws.com"
    }

Operating system and Environment details

macOS 12.4, M1 processor

Log Fragments and crash.log files

* Post-processor failed: failed to parse the ECR URL: 317215053986.dkr.ecr.us-east-2.amazonaws.com
parse "317215053986.dkr.ecr.us-east-2.amazonaws.com": invalid URI for request
it should be either of the form `public.ecr.aws/<registry_alias>/<registry_name>` or `<account number>.dkr.ecr.<region>.amazonaws.com`

Unable to push to repository due to "manifest invalid" error.

This issue was originally opened by @madelaney as hashicorp/packer#10879. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

When filing a bug, please include the following headings if possible. Any
example text in this template can be deleted.

Overview of the Issue

We have a number of packer configuration files (json), that builds a container but when we try to push to our internal registry we get an error of: docker (docker-push): manifest invalid: manifest invalid.

Reproduction Steps

Just run packer build <gist file> on Mac OS (Big Sur). I tried to reproduce this on Linux (Ubuntu 18.04) using Packer 1.7.2 and I could not but a coworker could reproduce this on another Mac OS (Big Sur) machiune.

Packer version

Tested with Packer 1.7.2, 1.6.0, and 1.5.6.

Simplified Packer Buildfile

A sample json file has been placed here

Operating system and Environment details

Mac OS X (amd64), Big Sur (11.2.2)

Log Fragments and crash.log files

From our internal registry (Artifactory) we see this in the logs:

2021-04-07T12:04:18.575Z [jfrt ] [WARN ] [7b02eb14f7c5ddb6] [.r.ArtifactoryResponseBase:125] [-nio-8081-exec-14780] - Sending HTTP error code 400: Property key: docker.label.desktop.docker.io/binds/0/SourceKind is invalid due to Name must start with a letter and cannot contain whitespace or special characters
2021-04-07T12:04:18.577Z [jfrt ] [ERROR] [7b02eb14f7c5ddb6] [h.DockerManifestPutHandler:261] [-nio-8081-exec-14780] - Error uploading manifest: 'Property key: docker.label.desktop.docker.io/binds/0/SourceKind is invalid due to Name must start with a letter and cannot contain whitespace or special characters'

From the packer run we see this:

    docker (docker-push): e5307c768962: Preparing
    docker (docker-push): fa728254b20e: Preparing
    docker (docker-push): 33a48e9ca2ba: Preparing
    docker (docker-push): 8aad4316c1e3: Preparing
    docker (docker-push): 9f7793952dca: Preparing
    docker (docker-push): 2bb84c5f5766: Preparing
    docker (docker-push): 8ea3b23f387b: Preparing
    docker (docker-push): 2bb84c5f5766: Waiting
    docker (docker-push): 8ea3b23f387b: Waiting
    docker (docker-push): 8aad4316c1e3: Layer already exists
    docker (docker-push): 33a48e9ca2ba: Layer already exists
    docker (docker-push): 9f7793952dca: Layer already exists
    docker (docker-push): fa728254b20e: Layer already exists
    docker (docker-push): 2bb84c5f5766: Layer already exists
    docker (docker-push): 8ea3b23f387b: Layer already exists
    docker (docker-push): e5307c768962: Pushed
    docker (docker-push): manifest invalid: manifest invalid
2021/04/07 08:05:58 packer-post-processor-docker-push plugin: error: Bad exit status: 1
2021/04/07 08:05:58 [INFO] (telemetry) ending docker-push

* Post-processor failed: Bad exit status: 1
==> Wait completed after 10 seconds 179 milliseconds
2021/04/07 08:05:58 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
Build 'docker' errored after 10 seconds 178 milliseconds: 1 error(s) occurred:
2021/04/07 08:05:58 machine readable: docker,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Bad exit status: 1"}


* Post-processor failed: Bad exit status: 1
==> Builds finished but no artifacts were created.
2021/04/07 08:05:58 [INFO] (telemetry) Finalizing.
* Post-processor failed: Bad exit status: 1

==> Wait completed after 10 seconds 179 milliseconds

FEATURE: Allow Docker images to be built remotely instead of on the host

This issue was originally opened by @dnk8n as hashicorp/packer#7963. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Why? For the same reason ebs-builder doesn't install everything locally and leave you with an AMI on your own dev machine, then push to Amazon.

I speak specifically about Amazon AWS, but other cloud providers should also apply (so this is a general request which applies to all builders)

Use-case: The host is often a docker container with minimal resources, using docker image: hashicorp/packer:1.4.2 for example. In many cases the docker run command is locked down as well.

It would be great if you could allow a similar workflow to ebs-builder, except end up with a Docker image being pushed instead of an AMI. That is, instead of provisioning locally, a temporary instance is created in ones' own VPC to do the heavy lifting.

Currently, everything is in place to allow that (if Packer and Docker are pre-installed on the remote instance which does the provisioning), except you are forced to create an AMI you don't care about.

Another use cases, say I need 8GB of RAM to compile something, then it would be useful to be able to use Packer's functionalities to use a tiny 'pilot' CI/CD runner to just run packer, with all heavy lifting done in one's own VPC.

Docker squash support

Seeing that docker build --squash is a experimental feature included in vanilla docker, it would be great if packer could support that option as well, instead of having to currently rely on external tools.

Docker provisioner caching

This issue was originally opened by @droopy4096 as hashicorp/packer#9758. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

Description

as per issue hashicorp/packer#1995 packer lacks the ability to use Docker cache (or any cache) for image building

Use Case(s)

At the moment I'm refactoring Microsoft Azure Pipeline Agent packer template to build container images (vs VM images) and finding the process to be overly tedious as build itself takes about 30min and with failures happening closer to the end of the build one has to spend extra 30min starting from start instead of recycling what already had been successfully built. There is no way to iterate fast without some sort of caching of all the "good" steps/layers.

Potential configuration

potentially having an option --with-cache=XXX save hashes for each step as tarballs (or other kind of blobs) under the XXX directory, along with the metadata for that step. upon re-run packer should be able to determine existence of cache for each step and use that.

Potential References

/bin/sh: 0: Can't open node

This issue was originally opened by @floydspace in hashicorp/packer#11150 and has been migrated to this repository. The original issue description is below.

Hi dear HashiCorp team.

I'm first day user of packer, willing to figure out how does it work. But it's already midnight and I cannot crack an error /bin/sh: 0: Can't open node when I run a container.

here is my pkg.hcl

source "docker" "example" {
  image = "node:14"
  commit = true
  changes = [
    "WORKDIR /app",
    "CMD [ \"node\", \"server.js\" ]"
  ]
}

build {
  sources = ["source.docker.example"]

  provisioner "file" {
    sources = ["./server.js"]
    destination = "/tmp/"
  }

  provisioner "shell" {
    inline = [
      "mkdir /app",
      "mv -f /tmp/server.js /app/"
    ]
  }

  post-processor "docker-tag" {
    repository = "local/node"
    tags = ["latest"]
  }
}

so I build it using command packer build pkr.hcl
and then running docker run -it --rm local/node:latest which fails with the error

thank you

Packer docker builder does not reset ENTRYPOINT properly

This issue was originally opened by @bertramn as hashicorp/packer#7487. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

We are using packer to build a container with a parent that includes an entrypoint. In the packer build we "reset" entrypoint with ENTRYPOINT []. Unfortunately the container is being committed with the entrypoint of the parent still in place.

Doing this in a Dockerfile gets rid of the entrypoint from parent:

FROM ep-issue:parent
ENTRYPOINT []
CMD []

While the same in packer version does not:

{
  "builders": [
    {
      "type": "docker",
      "image": "ep-issue:parent",
      "pull": "false",
      "commit": "true",
      "changes": [
        "ENTRYPOINT []",
        "CMD []"
      ]
    }
  ],
  "post-processors": [
    {
      "type": "docker-tag",
      "repository": "ep-issue",
      "tag": "child2"
    }
  ]
}

Packer version v1.3.5
Host platform MacOS High Sierra
Gist of packer-entrypoint-build.log
Gist of the simple script to reproduce the bug

Post-Processor: Delete docker image in local build machine after pushing to remote registry

This issue was originally opened by @karthik101 as hashicorp/packer#5361. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

I do not want docker images piling up in my jenkins-slave. Is there any way to not commit images to local build machine and able to push docker image to remote registry?

I tried with "export" in builder and "docker-import" in post-processor but its still keeps a copy.

Thanks,
Karthik

hashicorp / packer-plugin-docker Goto Github PK

packer-plugin-docker's Introduction

Packer Plugin Docker

Installation

Using pre-built releases

Using the packer init command

Manual installation

From Sources

Configuration

Contributing

packer-plugin-docker's People

Contributors

Stargazers

Watchers

Forkers

packer-plugin-docker's Issues

Community Note

Description

Use Case(s)

Potential configuration

Potential References

Overview of the Issue

Plugin and Packer version

Overview of the Issue

Reproduction Steps

Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Description

Use Case(s)

Overview of the Issue

Reproduction Steps

Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Reproduction Steps

Plugin and Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Reproduction Steps

Plugin and Packer version

Simplified Packer Buildfile

Operating system and Environment details

Description

Use Case(s)

Potential configuration

Potential References

Overview of the Issue

Reproduction Steps

Plugin and Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Reproduction Steps

Plugin and Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Container tmp permisions

Container exec user UID:GID

Packer build user UID:GID

Reproduction Steps

Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Reproduction Steps

Plugin and Packer version

Simplified Packer Buildfile

Operating system and Environment details

Log Fragments and crash.log files

Overview of the Issue

Using the `packer init` command