feature request: Using a tagged container as start of a new job. about docker-plugin HOT 7 CLOSED

Not sure if it directly helps, but the token macro ${DOCKER_CONTAINERID} will resolve to the build's container ID within the context of a build, you might be able to hack something up with that.

What might be hard is if you want those 'steps' to be jenkins slaves based on the output of the previous stage. I think that would be cool - but the problem is Jenkins launches slaves effectively by 'label' rather than 'specific image' -- it provisions machines in the background, then assigns jobs to them - not sure if there's a way around that..

from docker-plugin.

Indeed, for the time being I am attempting to set it up manually with the help of 'Docker build step plugin' which may give me the freedom I need to do this.

from docker-plugin.

Hit a wall trying to use the 'Docker build step plugin' as well, but I am sure it can be realized using that fine plugin in the near future.

For the moment (for those interested) I realized above utilizing the power of the https://wiki.jenkins-ci.org/display/JENKINS/inheritance-plugin (I use version 1.5.0). Creating two abstract jobs, docker_before and docker_after and making sure the are run before and after for each task added.

docker_before has one build step that looks like this:

if [ -n "${PREVIOUS_STEP}" ]; then
  EXISTING_TAG_ID=$(docker images | grep "${PREVIOUS_STEP}-${REVISION_HASH}" | awk '{print $3}')
  if [ -z "${EXISTING_TAG_ID}" ]; then
    echo "FAIL, not run ${PREVIOUS_STEP} step yet"
    exit 1
  fi
else
  EXISTING_TAG_ID=${DOCKER_IMAGE_ID}:latest
fi
export DOCKER_IMAGE_WORKSPACE=/home/jenkins/workspace
export DOCKER_CONTAINER_ID=$(docker run --dns=${DNS} -P -d ${EXISTING_TAG_ID} /usr/sbin/sshd -D)
export DOCKER_CONTAINER_PORT=$(docker inspect ${DOCKER_CONTAINER_ID} | grep HostPort |head -1  | cut -d "\"" -f 4)

echo "export DOCKER_CONTAINER_ID=${DOCKER_CONTAINER_ID}" > ./env-${BUILD_TAG}
echo "export DOCKER_CONTAINER_PORT=${DOCKER_CONTAINER_PORT}" >> ./env-${BUILD_TAG}
echo "export DOCKER_IMAGE_WORKSPACE=${DOCKER_IMAGE_WORKSPACE}" >> ./env-${BUILD_TAG}
echo "export STEP_NAME=start" >> ./env-${BUILD_TAG}
# check login
ssh jenkins@${DOCKER_HOST_IP} -p ${DOCKER_CONTAINER_PORT} -o "StrictHostKeyChecking no" bash -c "date" || \
sleep 3 ; ssh jenkins@${DOCKER_HOST_IP} -p ${DOCKER_CONTAINER_PORT} -o "StrictHostKeyChecking no" bash -c "date"

and docker_after looks like this:

source ./env-${BUILD_TAG} 

# find existing tagged image
EXISTING_TAG_ID=$(docker images | grep "${STEP_NAME}-${REVISION_HASH}" | awk '{print $3}')

docker stop ${DOCKER_CONTAINER_ID}
if [ "${DO_NOT_COMMIT}" == "0" ]; then
  docker commit -m "${STEP_NAME}-${REVISION_HASH}" ${DOCKER_CONTAINER_ID} ${DOCKER_IMAGE_ID}:${STEP_NAME}-${REVISION_HASH}
fi
docker rm ${DOCKER_CONTAINER_ID}

# remove old existing tagged image
if [ -n "${EXISTING_TAG_ID}" ]; then
  docker rmi ${EXISTING_TAG_ID} || true
fi

rm -rf ./env-${BUILD_TAG}

the build step job then becomes something like this:

source ./env-${BUILD_TAG}

echo "export STEP_NAME=build" >> ./env-${BUILD_TAG}

banner "Build It" | cowsay -f turtle -n

ssh jenkins@${DOCKER_HOST_IP} -p ${DOCKER_CONTAINER_PORT} -o "StrictHostKeyChecking no" \
"${DOCKER_IMAGE_WORKSPACE}/ops/bin/buildit"

environment variables you see here, that are no defined, are defined at the 'manage jenkins' level or are passed in as parameters (DO_NOT_COMMIT, PREVIOUS_STEP, REVISION_HASH)

from docker-plugin.

Would it also be possible to use the previous container to do the next build?

For example: Run a build for the first time and it downloads dependencies etc. and runs the build. The container is then committed to a common docker repository. The next host to build then pulls the committed previous build image, with the dependencies already downloaded. This helps to increase logfile readability by not having millions of DOWNLOADED: lines and increases build speed.

from docker-plugin.

@magnayn is this related to your jenkins PR?

from docker-plugin.

if you want multiple steps to run within the same container context, why split this into multiple jobs ? Also, sounds to me what you're looking for a reusing a volume, not container

from docker-plugin.

please reopen if still relevant

from docker-plugin.