Comments (4)
it seems that in Singularity, the environment variables made in %environment
are not available in %post
. am i correct?
i tested this using a small Singularity file
# Test whether updated environment variables like PATH are available in %post.
Bootstrap: docker
From: fedora:36
%environment
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"
%post
echo $PATH
echo $NEWVAR
and during the build, I get the following output, which shows that the updated PATH
variable is not available in %post
.
INFO: Starting build...
Getting image source signatures
Copying blob c7bef7d09442 skipped: already exists
Copying config 4d934e266f done
Writing manifest to image destination
Storing signatures
2023/10/25 10:38:22 info unpack layer: sha256:c7bef7d09442a4c4deef8e8120027e7fa3f163c7b5a0afedc3f2c4c2e7652e1c
INFO: Running post scriptlet
+ echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ echo
INFO: Adding environment to container
INFO: Creating SIF file...
INFO: Build complete: foobar
for help in debugging, here are the Dockerfile and Singularity file for the command
singularity run docker://repronim/neurodocker:latest generate singularity \
--pkg-manager yum \
--base-image fedora:36 \
--afni method=binaries version=latest install_r_pkgs=true
Dockerfile
# Generated by Neurodocker and Reproenv.
FROM fedora:36
ENV PATH="/opt/afni-latest:$PATH" \
AFNI_PLUGINPATH="/opt/afni-latest"
RUN yum install -y -q \
R \
cmake \
curl \
ed \
gsl \
libGLU \
libXp \
libXpm \
libcurl-devel \
libgomp \
libjpeg-turbo-devel \
libpng12 \
mesa-dri-drivers \
mesa-dri-drivers \
mesa-libGLw \
ncurses-compat-libs \
netpbm-progs \
openmotif \
openssl-devel \
python-is-python3 \
python3-pip \
tcsh \
udunits2-devel \
unzip \
wget \
which \
which \
xorg-x11-fonts-misc \
xorg-x11-server-Xvfb \
&& yum clean all \
&& rm -rf /var/cache/yum/* \
&& gsl_path="$(find / -name 'libgsl.so.??' || printf '')" \
&& if [ -n "$gsl_path" ]; then \
ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; \
fi \
&& ldconfig \
&& mkdir -p /opt/afni-latest \
&& echo "Downloading AFNI ..." \
&& curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \
| tar -xz -C /opt/afni-latest --strip-components 1 \
&& yum install -y -q \
R-devel \
&& yum clean all \
&& rm -rf /var/cache/yum/* \
&& rPkgsInstall -pkgs ALL
# Save specification to JSON.
RUN printf '{ \
"pkg_manager": "yum", \
"existing_users": [ \
"root" \
], \
"instructions": [ \
{ \
"name": "from_", \
"kwds": { \
"base_image": "fedora:36" \
} \
}, \
{ \
"name": "env", \
"kwds": { \
"PATH": "/opt/afni-latest:$PATH", \
"AFNI_PLUGINPATH": "/opt/afni-latest" \
} \
}, \
{ \
"name": "run", \
"kwds": { \
"command": "yum install -y -q \\\\\\n R \\\\\\n cmake \\\\\\n curl \\\\\\n ed \\\\\\n gsl \\\\\\n libGLU \\\\\\n libXp \\\\\\n libXpm \\\\\\n libcurl-devel \\\\\\n libgomp \\\\\\n libjpeg-turbo-devel \\\\\\n libpng12 \\\\\\n mesa-dri-drivers \\\\\\n mesa-dri-drivers \\\\\\n mesa-libGLw \\\\\\n ncurses-compat-libs \\\\\\n netpbm-progs \\\\\\n openmotif \\\\\\n openssl-devel \\\\\\n python-is-python3 \\\\\\n python3-pip \\\\\\n tcsh \\\\\\n udunits2-devel \\\\\\n unzip \\\\\\n wget \\\\\\n which \\\\\\n which \\\\\\n xorg-x11-fonts-misc \\\\\\n xorg-x11-server-Xvfb\\nyum clean all\\nrm -rf /var/cache/yum/*\\n\\ngsl_path=\\"$\(find / -name '"'"'libgsl.so.??'"'"' || printf '"'"''"'"'\)\\"\\nif [ -n \\"$gsl_path\\" ]; then \\\\\\n ln -sfv \\"$gsl_path\\" \\"$\(dirname $gsl_path\)/libgsl.so.0\\"; \\\\\\nfi\\nldconfig\\nmkdir -p /opt/afni-latest\\necho \\"Downloading AFNI ...\\"\\ncurl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \\\\\\n| tar -xz -C /opt/afni-latest --strip-components 1\\n yum install -y -q \\\\\\n R-devel\\nyum clean all\\nrm -rf /var/cache/yum/*\\n rPkgsInstall -pkgs ALL" \
} \
} \
] \
}' > /.reproenv.json
# End saving to specification to JSON.
Singularity
# Generated by Neurodocker and Reproenv.
Bootstrap: docker
From: fedora:36
%environment
export PATH="/opt/afni-latest:$PATH"
export AFNI_PLUGINPATH="/opt/afni-latest"
%post
yum install -y -q \
R \
cmake \
curl \
ed \
gsl \
libGLU \
libXp \
libXpm \
libcurl-devel \
libgomp \
libjpeg-turbo-devel \
libpng12 \
mesa-dri-drivers \
mesa-dri-drivers \
mesa-libGLw \
ncurses-compat-libs \
netpbm-progs \
openmotif \
openssl-devel \
python-is-python3 \
python3-pip \
tcsh \
udunits2-devel \
unzip \
wget \
which \
which \
xorg-x11-fonts-misc \
xorg-x11-server-Xvfb
yum clean all
rm -rf /var/cache/yum/*
gsl_path="$(find / -name 'libgsl.so.??' || printf '')"
if [ -n "$gsl_path" ]; then \
ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; \
fi
ldconfig
mkdir -p /opt/afni-latest
echo "Downloading AFNI ..."
curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \
| tar -xz -C /opt/afni-latest --strip-components 1
yum install -y -q \
R-devel
yum clean all
rm -rf /var/cache/yum/*
rPkgsInstall -pkgs ALL
# Save specification to JSON.
printf '{ \
"pkg_manager": "yum", \
"existing_users": [ \
"root" \
], \
"instructions": [ \
{ \
"name": "from_", \
"kwds": { \
"base_image": "fedora:36" \
} \
}, \
{ \
"name": "env", \
"kwds": { \
"PATH": "/opt/afni-latest:$PATH", \
"AFNI_PLUGINPATH": "/opt/afni-latest" \
} \
}, \
{ \
"name": "run", \
"kwds": { \
"command": "yum install -y -q \\\\\\n R \\\\\\n cmake \\\\\\n curl \\\\\\n ed \\\\\\n gsl \\\\\\n libGLU \\\\\\n libXp \\\\\\n libXpm \\\\\\n libcurl-devel \\\\\\n libgomp \\\\\\n libjpeg-turbo-devel \\\\\\n libpng12 \\\\\\n mesa-dri-drivers \\\\\\n mesa-dri-drivers \\\\\\n mesa-libGLw \\\\\\n ncurses-compat-libs \\\\\\n netpbm-progs \\\\\\n openmotif \\\\\\n openssl-devel \\\\\\n python-is-python3 \\\\\\n python3-pip \\\\\\n tcsh \\\\\\n udunits2-devel \\\\\\n unzip \\\\\\n wget \\\\\\n which \\\\\\n which \\\\\\n xorg-x11-fonts-misc \\\\\\n xorg-x11-server-Xvfb\\nyum clean all\\nrm -rf /var/cache/yum/*\\n\\ngsl_path=\\"$\(find / -name '"'"'libgsl.so.??'"'"' || printf '"'"''"'"'\)\\"\\nif [ -n \\"$gsl_path\\" ]; then \\\\\\n ln -sfv \\"$gsl_path\\" \\"$\(dirname $gsl_path\)/libgsl.so.0\\"; \\\\\\nfi\\nldconfig\\nmkdir -p /opt/afni-latest\\necho \\"Downloading AFNI ...\\"\\ncurl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \\\\\\n| tar -xz -C /opt/afni-latest --strip-components 1\\n yum install -y -q \\\\\\n R-devel\\nyum clean all\\nrm -rf /var/cache/yum/*\\n rPkgsInstall -pkgs ALL" \
} \
} \
] \
}' > /.reproenv.json
# End saving to specification to JSON.
from neurodocker.
indeed, singularity does not make %environment
variables available in %post
by default...
related:
i think the solution would be to add . $SINGULARITY_ENVIRONMENT
to the beginning of %post
.
@stebo85 - what do you think?
if apptainer / singularity developers are reading, then i would like to say that i find this behavior highly unexpected.
from neurodocker.
Dear @kaczmarj - great find - that explains the problems.
I tried adding . $SINGULARITY_ENVIRONMENT
to the %post section, but this results in:
INFO: Running post scriptlet
+ . /.singularity.d/env/91-environment.sh
/.post.script: line 1: /.singularity.d/env/91-environment.sh: No such file or directory
FATAL: While performing build: while running engine: exit status 1
I know it's not elegant, but would anything else speak against re-exporting the environment variables in the post section?
Bootstrap: docker
From: fedora:36
%environment
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"
%post
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"
echo $PATH
echo $NEWVAR
at least it works :?
INFO: Running post scriptlet
+ export PATH=/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export NEWVAR=ThisIsATest
+ NEWVAR=ThisIsATest
+ echo /opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ echo ThisIsATest
ThisIsATest
@vsoch - could you maybe have quick look at what we are getting wrong here?
Kind regards
Steffen
from neurodocker.
I know it's not elegant, but would anything else speak against re-exporting the environment variables in the post section?
fine with me if there's no better solution.
from neurodocker.
Related Issues (20)
- matlab runtime 2023a
- freesurfer 7.4.1 HOT 2
- add instructions for making a new template HOT 2
- install all fails HOT 1
- [MAINT] imporave readability of type hinting
- [MAINT] split lint / formatting checks from tests in CI
- unclear why python 3.10 is more required to pass than latest python HOT 2
- do not test the builds on ubuntu 16:04 and debian stretch HOT 1
- close stale issues HOT 2
- `pip install -e .dev` fails in CI HOT 3
- add install of bids validator HOT 1
- add possibility to only "dispatch" the bootstrap workflow for a given software HOT 2
- Ensure examples in doc can actually build
- [DOC] add a table of supported softwares
- fix nipype example
- can't build afni with R HOT 7
- jq from source fails HOT 1
- Error when trying to add output filename during generate docker command HOT 2
- Update docker image and pip versions to latest HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neurodocker.