Code Monkey home page Code Monkey logo

cmssw-osenv's People

Contributors

kpedro88 avatar smuzaffar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

kpedro88

cmssw-osenv's Issues

Accessing aliases inside Singularity

Aliases are limited in bash (and related shells). As far as I can tell, there is absolutely no way to make a construct like this "work":

sh -c 'source ../cmsset_default.sh; /bin/bash'

where "work" is defined to mean that everything, including aliases, is passed to the (interactive) subshell. (shopt -s expand_aliases is not effective in this case, because the alias is defined before the subshell exists.) This has become more noticeable since the switch to el8 as the production OS. Since many of us do not have el8 machines yet, we are more often using the el8 container interactively, whereas previously (at least for me) the main usage of CMSSW containers was for specific commands or batch jobs.

I could only find one way to get this to work solely by modifying cmssw-env in this repo: master...kpedro88:here-string-tty

However, this is a bit clunky, as it displays on the screen (and puts into the command history) the following:

[ "slc7_amd64" != "$(./cmsos)" ] && export SCRAM_ARCH=; source ./../cmsset_default.sh ; exec </dev/tty

There is an alternative: use functions instead of aliases (as most Bash experts seem to recommend in the modern era). With a few changes in cmsset_default.sh, everything works: cms-sw/cms-common@master...kpedro88:use-function

Therefore, I propose changing cmsset_default.sh; please provide any feedback if there is a reason not to do so.

Mounting of /eos

These scripts automatically mount /eos. This might not be desirable in general:

It would be nice to have some toggle to turn this off or on. Personally, I would prefer if the default were "off", but I am open to rebuttals on that point before making a PR.

cmssw-env runs 'ls /cvmfs/projects.cern.ch' which causes issues outside cern.ch network.

The cvmfs service keeps switching proxies trying to connect to projects.cern.ch and eventually fails. This causes the cmssw-env script to timeout for a few seconds when starting.

Jul  1 12:35:31 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/projects.cern.ch to http://cvmfs-s1bnl.opensciencegrid.org:8000/cvmfs/projects.cern.ch (host returned HTTP error)e
Jul  1 12:35:31 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-s1bnl.opensciencegrid.org:8000/cvmfs/projects.cern.ch to http://cernvmfs.gridpp.rl.ac.uk:8000/cvmfs/projects.cern.ch (host returned HTTP error)r
Jul  1 12:35:31 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cernvmfs.gridpp.rl.ac.uk:8000/cvmfs/projects.cern.ch to http://cvmfs-stratum-one.cern.ch:8000/cvmfs/projects.cern.ch (host returned HTTP error)o
Jul  1 12:35:31 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-stratum-one.cern.ch:8000/cvmfs/projects.cern.ch to http://cvmfs-stratum-one.ihep.ac.cn:8000/cvmfs/projects.cern.ch (host returned HTTP error)t
Jul  1 12:35:32 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-stratum-one.ihep.ac.cn:8000/cvmfs/projects.cern.ch to http://cvmfsrep.grid.sinica.edu.tw:8000/cvmfs/projects.cern.ch (host returned HTTP error)c
Jul  1 12:35:42 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching proxy from http://131.225.152.34:3128 to http://131.225.188.245:3126 (failed proxy)a
Jul  1 12:35:47 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching proxy from http://131.225.188.245:3126 to http://131.225.188.246:3126 (failed proxy)f
Jul  1 12:35:52 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching proxy from http://131.225.188.246:3126 to http://128.142.248.156:3126 (failed proxy)f
Jul  1 12:35:58 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching proxy from http://128.142.248.156:3126 to http://128.142.161.84:3126 (failed proxy)a
Jul  1 12:36:04 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching proxy from http://128.142.161.84:3126 to DIRECT (failed proxy)
Jul  1 12:36:14 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfsrep.grid.sinica.edu.tw:8000/cvmfs/projects.cern.ch to http://cvmfs-s1.hpc.swin.edu.au:8000/cvmfs/projects.cern.ch (host serving data too slowly)h
Jul  1 12:36:15 gartung-desktop cvmfs2[13815]: (projects.cern.ch) failed to download repository manifest (9 - host returned HTTP error)
Jul  1 12:36:15 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-s1.hpc.swin.edu.au:8000/cvmfs/projects.cern.ch to http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/projects.cern.ch (host returned HTTP error)o
Jul  1 12:36:15 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/projects.cern.ch to http://cvmfs-s1bnl.opensciencegrid.org:8000/cvmfs/projects.cern.ch (host returned HTTP error)e
Jul  1 12:36:15 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-s1bnl.opensciencegrid.org:8000/cvmfs/projects.cern.ch to http://cernvmfs.gridpp.rl.ac.uk:8000/cvmfs/projects.cern.ch (host returned HTTP error)r
Jul  1 12:36:16 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cernvmfs.gridpp.rl.ac.uk:8000/cvmfs/projects.cern.ch to http://cvmfs-stratum-one.cern.ch:8000/cvmfs/projects.cern.ch (host returned HTTP error)o
Jul  1 12:36:16 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-stratum-one.cern.ch:8000/cvmfs/projects.cern.ch to http://cvmfs-stratum-one.ihep.ac.cn:8000/cvmfs/projects.cern.ch (host returned HTTP error)t
Jul  1 12:36:17 gartung-desktop cvmfs2[13815]: (projects.cern.ch) switching host from http://cvmfs-stratum-one.ihep.ac.cn:8000/cvmfs/projects.cern.ch to http://cvmfsrep.grid.sinica.edu.tw:8000/cvmfs/projects.cern.ch (host returned HTTP error)c
Jul  1 12:36:37 gartung-desktop cvmfs2[13815]: (projects.cern.ch) failed to fetch file catalog at projects.cern.ch:/ (hash: 0000000000000000000000000000000000000000, error 15 [host serving data too slowly])0
Jul  1 12:36:37 gartung-desktop cvmfs2[13815]: (projects.cern.ch) Failed to initialize root file catalog (16 - file catalog failure)

xdg-open missing from cmssw-el8

When trying to launch a ROOT TBrowser in the cmssw-el8 container, I get the following error message:

sh: xdg-open: command not found

Can the container be updated to include this (and any other basic graphics packages)? Being able to inspect the output of test workflows is important for development.

locales missing from el8 and (some) rhel8 containers

It seems the CMSSW Singularity containers for el8 and (in certain cases) rhel8 are lacking many locales, in particular, en_US.utf-8. For some reason, this causes a problem for icc, the Intel C/C++ Classic Compiler that is now included for free with the Intel oneAPI HPC Toolkit. This compiler could very well be installed on one's host system, and it should be usable for testing purposes in the container. Instead, I find that icc generally exits with an error when one tries to use it for compiling in the container.

I have identified a few workarounds, such as setting the -no-multibyte-chars option for icc, or doing export LC_ALL=C.utf-8 in the shell. However, I believe it would be simpler (and therefore preferable) to make more locales available in the image, as described here: apptainer/singularity#3765 (the last post in particular).

This is worth opening an issue, I think, and I am hoping this is the right place to do it, even though the issue is perhaps best corrected in the images themselves, and not in the wrapper scripts. You can see that there is an issue in the images by starting up the following two containers with singularity exec XXX /bin/bash and comparing the output of locale -a:

XXX=/cvmfs/unpacked.cern.ch/registry.hub.docker.com/cmssw/el8:x86_64
XXX=/cvmfs/unpacked.cern.ch/registry.hub.docker.com/cmssw/cms:rhel8-m20220516

The latter has many, many more locales. But note that /cvmfs/unpacked.cern.ch/registry.hub.docker.com/cmssw/cms:rhel8-m (without the additional numbers) looks more like el8:x86_46 in having just a few locales available. Anyway, in the cms:rhel8-m20220516 container, I find that icc works fine, with no need to export LC_ALL=C.utf-8 (e.g.).

Without one of the workarounds, the precise error message from icc is: 'Catastrophic error: could not set locale "" to allow processing of multibyte characters'

Install perf in Singularity images

Linux perf can be used to generate output that might be convertable to Igprof format. However I don't find perf in the EL8 Singularity imaged when I run cmssw-el8.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.