Code Monkey home page Code Monkey logo

container-definitions's Introduction

Container Definitions

This repository contains Bazel targets for Google-maintained common container definitions and their dependencies. Each folder under this repo is a standalone Bazel project which builds deterministic and reproducible containers at every commit.

Source Code Headers

Every file containing source code must include copyright and license information.

Apache header:

Copyright 2018 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

container-definitions's People

Contributors

alex1545 avatar davegay avatar eytankidron avatar nlopezgi avatar rbe-toolchains-robot avatar smukherj1 avatar xingao267 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

container-definitions's Issues

bazel container fails with "FATAL: mkdir('/tmp/build_output'): (error: 13): Permission denied"

I'm trying to use the l.gcr.io/google/bazel:latest container as described by the Getting Started with Bazel Docker Container page. I'm suspecting this is a problem with the doc, but it might be a bug. I'm unfamiliar with the way this container is built and I'm unable to debug it further.

The command as documented fails with FATAL: mkdir('/tmp/build_output'): (error: 13): Permission denied

docker run \
  -e USER="$(id -u)" \
  -u="$(id -u)" \
  -v /tmp/src/workspace:/src/workspace \
  -v /tmp/build_output:/tmp/build_output \
  -w /src/workspace \
  l.gcr.io/google/bazel:latest \
  --output_user_root=/tmp/build_output \
  build //absl/...

(well, I tweaked from the doc a bit as having to create /src needs root - I did create the directories used here)

This command is dubious as the selinux labels are not taken into consideration. So one would expect problems reading or writing data at /src/workspace or /tmp/build_output anyway. Since /tmp/build_output is bind mounted into the container, it then exists and is a mount, it is not surprising that mkdir without -p would also fail. The problem might also be related to #4677.

edit: the mkdir must be bazel processing the --output_user_root parameter. The doc for docker run --output_user_root mentions the directory must not exist and be owned by the calling user. So this might be due to uid mapping with the container.

Tweaking commands works though..

Here we run internally in the container as root, and use the Z options to set the selinux labels.
Here I assume the mkdir is still happening internally, running as root might work around the problem. This build works, though ownership of the files on the docker host will be an issue to deal with.

docker run \
  -v /tmp/src/workspace:/src/workspace:Z \
  -v /tmp/build_output:/tmp/build_output:Z \
  -w /src/workspace \
  l.gcr.io/google/bazel:latest \
  --output_user_root=/tmp/build_output \
  build //absl/...

# Build works!

Another option is to leave off the bind mount for /tmp/build_output. Not ideal, but you could still use docker cp or docker mount to get the stuff you need out of the container.

docker run \
  -e USER="$(id -u)" \
  -u="$(id -u)" \
  -v /tmp/src/workspace:/src/workspace:Z \
  -w /src/workspace \
  l.gcr.io/google/bazel:latest \
  --output_user_root=/tmp/build_output \
  build //absl/...

# Build works!

So if we pick one of those (probably the root version), maybe this boils down to a doc issue. I'm willing to contribute an update.

Other things tried

I tried changing the bind mount to bind mount over /tmp (-v /tmp/build_output:/tmp ), thinking the mkdir would work. But still got the error. Also used a real newly created volume (-v absl-workspace:/tmp/build_output \), and that also failed. Tried changing --output_user_root to a subdirectory of the bind mount, same error code 13.

I had set out to revisit intra-container permission problems I had seen last year like #4677. Since I am able to run the container now, perhaps the problem has been addressed. That or this mkdir problem is now hiding that problem in some cases.

I tested with the same results on RHEL 8.2 with podman 1.93 and on Ubuntu 20.04.2 with Docker 19.03.8. The bazel image was

l.gcr.io/google/bazel latest 5cac8433a9d7 51 years ago 1.62GB

Replace Ubuntu 16.04 LTS in container definitions (end of standard support)

Ubuntu 16.04 LTS reached the end of standard support in April 2021. Updates are only provided through Extended Security Maintenance (ESM) which must be purchased from Canonical1. The latest publicly-available version of Ubuntu 16.04 LTS is affected by 343 CVEs.

The container definitions using Ubuntu 16.04 LTS should be changed to target a newer release. Base images are currently available for Ubuntu 18.04 LTS and 20.04 LTS, which remain under standard support.

Footnotes

  1. ESM is free for personal use on up to 3 systems. For use on GCP or other public clouds, Ubuntu Pro is a paid image that includes ESM.

Permission issue in "Getting started with Bazel Docker Container" guide

Hi. I've just followed this guide and am seeing a permissions error.

https://github.com/bazelbuild/bazel/blob/master/site/docs/bazel-container.md
https://docs.bazel.build/versions/master/bazel-container.html

Specifically, when I run:

# this doesn't work
git clone https://github.com/abseil/abseil-cpp.git /src/workspace
mkdir -p /tmp/build_output/
docker run \
  -e USER="$(id -u)" \
  -u="$(id -u)" \
  -v /src/workspace:/src/workspace \
  -v /tmp/build_output:/tmp/build_output \
  -w /src/workspace \
  l.gcr.io/google/bazel:latest \
  --output_user_root=/tmp/build_output \
  build //absl/...

I get the following permission error:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"/usr/local/bin/bazel\": stat /usr/local/bin/bazel: permission denied": unknown.
ERRO[0007] error waiting for container: context canceled 

It seems that the owner of /usr/local/bin/bazel inside the container is root - so setting the user inside the container to a non-root user means the binary can't be executed.

Currently my workaround is to create /tmp/build_output in the host system as the root user. This works fine as then the user inside the container is also root (docker's default user if you don't specify --user="...").

# This works fine...
git clone https://github.com/abseil/abseil-cpp.git /src/workspace
sudo mkdir -p /tmp/build_output/ ## need to use sudo or --output_user_root causes a different permissions error
docker run \
  -v /src/workspace:/src/workspace \
  -v /tmp/build_output:/tmp/build_output \
  -w /src/workspace \
  l.gcr.io/google/bazel:latest \
  --output_user_root=/tmp/build_output \
  build //absl/...

Ideally I'd like to run the container with --user="$(id -u)" as the guide suggests.

Is it possible for you to change how the container is build to enable this?

The goal: make it possible to run bazel build in the host system OR via the container and have all files created on the host system be exactly the same (including the same non-root permissions, so they can be used from the host).

This article explains docker shared permissions in case it's relevant:
https://vsupalov.com/docker-shared-permissions/

encounter error on building bazel image

/tmp/container-definitions/ubuntu1604_bazel [master|✔]
18:07 $ bazel build --verbose_failures   //:image
INFO: Analyzed target //:image (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/tmp/container-definitions/ubuntu1604_bazel/BUILD:32:1: error executing shell command: '/bin/bash -c tar cvf bazel-out/darwin-fastbuild/bin/image_intermediate-packages.tar --files-from /dev/null &&         for i in external/ubuntu1604_clang_debs/file/1586183502_clang_debs.tar external...' failed (Exit 1): bash failed: error executing command
  (cd /private/var/tmp/_bazel_kazimi/7ef70d95901386366ebd4a7cde5188b4/sandbox/darwin-sandbox/36/execroot/ubuntu1604_bazel && \
  exec env - \
  /bin/bash -c 'tar cvf bazel-out/darwin-fastbuild/bin/image_intermediate-packages.tar --files-from /dev/null &&         for i in external/ubuntu1604_clang_debs/file/1586183502_clang_debs.tar external/ubuntu1604_java_debs/file/1580247144_java_debs.tar external/ubuntu1604_python_debs/file/1587475877_python_debs.tar external/ubuntu1604_bazel_debs/file/1587472295_bazel_debs.tar external/ubuntu1604_docker_debs/file/1584020080_docker_debs.tar; do tar A --file=bazel-out/darwin-fastbuild/bin/image_intermediate-packages.tar $i; done')
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox bash failed: error executing command
  (cd /private/var/tmp/_bazel_kazimi/7ef70d95901386366ebd4a7cde5188b4/sandbox/darwin-sandbox/36/execroot/ubuntu1604_bazel && \
  exec env - \
  /bin/bash -c 'tar cvf bazel-out/darwin-fastbuild/bin/image_intermediate-packages.tar --files-from /dev/null &&         for i in external/ubuntu1604_clang_debs/file/1586183502_clang_debs.tar external/ubuntu1604_java_debs/file/1580247144_java_debs.tar external/ubuntu1604_python_debs/file/1587475877_python_debs.tar external/ubuntu1604_bazel_debs/file/1587472295_bazel_debs.tar external/ubuntu1604_docker_debs/file/1584020080_docker_debs.tar; do tar A --file=bazel-out/darwin-fastbuild/bin/image_intermediate-packages.tar $i; done')
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help
Target //:image failed to build
INFO: Elapsed time: 4.664s, Critical Path: 3.60s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

Networking QUESTION

Why do we need both MAC addresses and IP addresses to deliver a message from source to
destination? It may be useful to detail the passage of a message across a simple inter network to explain your answer.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.