Comments (10)
Looking at another discussion on a similar topic, the slowdowns are likely most prominent when the overlay2
storage driver is used.
from docker-images.
FWIW it looks like CircleCI also uses overlay2
. Definitely have seen some slowdown due to this personally, but not to the tune of 8mins. Probably closer to seconds. Though haven't been hit bad enough personally to need to measure. Wonder if there is something more going on in your case. Perhaps a slow OS FS?
Would be open to other options. The main challenge is that we need to make sure the UID of the user in the container matches that of the user outside the container. Otherwise we basically can't do anything with the mounted directories. Given how the feedstocks are setup, this includes everything from updating packages, doing the build, etc.
from docker-images.
We also use a linux-anvil inspired image for our internal CI, and since our I/O budget is a bit tight, this was slightly annoying. What we do is simply to chown the conda dir like:
# lucky group gets permissions to write in the conda dir
RUN groupadd -g 32767 lucky
RUN chgrp -R lucky /opt/conda && chmod -R g=u /opt/conda
And then add the unprivileged user to the "lucky" group each time we spin a container.
from docker-images.
@gqmelo, thoughts?
from docker-images.
I tried to reproduce the issue on Ubuntu 18.04, and after configuring docker to use overlay2
instead of aufs
it indeed slowed the build considerably. It took about 2.5 min just to execute the chown/chmod commands, so it seems realistic to say the 8 min
mentioned is caused by overlay2
.
By reading the issue mentioned before, there is still no fix for that, so the workaround suggested by @sdvillal seems good. It would also make the entrypoint
much simpler, as we could move a bunch of code to the Dockerfile
.
I can't see any downside for this solution. We would still be creating files on build_artifacts
with the host UID.
from docker-images.
We could also look at using fixuid
. Basically a tool intended to solve this problem.
from docker-images.
But it would face the same problem. From the README:
changes all file permissions for old dockeruser:dockergroup to
from docker-images.
RUN groupadd -g 32767 lucky
Is this number meaningful or is it arbitrary, @sdvillal?
from docker-images.
Arbitrary.
from docker-images.
#68 tackles this
from docker-images.
Related Issues (20)
- Add linux-anvil-cos7-x86_64-cuda HOT 1
- Update conda version to 4.8.4 HOT 1
- docker image condaforge/linux-anvil-comp7 failing with yum error
- Add mambaforge image
- move to azure HOT 4
- `/usr/local/cuda/bin` is not added to `$PATH` in `linux-anvil-ppc64le-cuda` HOT 13
- Including mamba in base images HOT 13
- ppc64le image builds failing HOT 3
- Use mamba HOT 6
- Instant segfault for linux-anvil-comp7 on WSL2 HOT 5
- centos images & user+mnt namespaces HOT 2
- Move CUDA 11.5 images to CentOS 7 HOT 1
- CUDA 11.0+ PPC & ARM builds failing HOT 6
- can we move all biz to userspace? HOT 5
- Scheduled runs haven't run in a while
- Latest anvil image installs incompatible conda and mamba versions HOT 1
- Updating QEMU HOT 5
- Drop armv7l bits
- Update qemu
- RUN conda install -n python-rest-1 -c conda-forge tiktoken not working complaining that package is not available. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-images.