Code Monkey home page Code Monkey logo

run-cvmfs-osg-eic-shell's Introduction

GitHub Action: eic/run-cvmfs-osg-eic-shell

linux

This GitHub Action executes user payload code inside a EIC shell environment, specified by the user.

Instructions

Prerequisites

This action depends on the user to call the companion action uses: cvmfs-contrib/github-action-cvmfs@v2 before using uses: eic/run-cvmfs-osg-eic-shell@v1, which will install CVMFS on the node. GitHub Actions currently do not support calling the action github-action-cvmfs from within run-cvmfs-osg-eic-shell, this needs to be done explicitly by the user.

Example

You can use this GitHub Action in a workflow in your own repository with uses: eic/run-cvmfs-osg-eic-shell@v1.

A minimal job example for GitHub-hosted runners of type ubuntu-latest:

jobs:
  run-eic-shell:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: cvmfs-contrib/github-action-cvmfs@v2
    - uses: eic/run-cvmfs-osg-eic-shell@v1
      with:
        platform-release: "jug_xl:nightly"
        run: |
          gcc --version
          which gcc
          eic-info

In this case the action will automatically resolve the correct container image (in this case jug_xl:nightly) and spawn an instance with Singularity from /cvmfs/singularity.opensciencegrid.org/.

The action mounts the checkout directory into the mentioned container and wraps the variable run in the script:

#!/usr/bin/env bash
export LC_ALL=C
set -Euo pipefail
trap 's=$?; echo "$0: Error on line "$LINENO": $BASH_COMMAND"; exit $s' ERR
IFS=$'\n\t'
set -e

source ${SETUP}

${RUN} # the multi-line variable specified in the action under run: |

which is executed in the container and thus giving the user an easy and direct access to run arbitrary code on top of an EIC shell.

Parameters

The following parameters are supported:

  • platform: EIC shell platform you are targeting (e.g. jug_xl)
  • release: EIC shell release you are targeting (e.g. 3.0-stable)
  • platform-release: EIC shell platform release string you are targeting (e.g. jug_xl:3.0-stable)
  • run: They payload code you want to execute on top of the view
  • setup: Initialization/Setup script for a view that sets the environment (e.g. /opt/detector/epic-main/bin/thisepic.sh)
  • sandbox-path: Path where the setup script for the custom view is location. By specifying this variable the auto-resolving of the view based on release and platform is disabled.
  • network_types: Network types to setup inside container. Defaults to bridge networking, but can be none to disable networking.
  • apptainer_version: Apptainer version to use. Defaults to latest, but can be any version such as v1.1.3.

Please be aware that you must use the combination of parameters release and platform together or use just the variable platform-release alone. These two options are given to enable more flexibility for the user to form their workflow with matrix expressions.

Minimal Example

There are minimal examples, which are also workflows in this repository in the subfolder .github/workflows/.

Limitations

The action will always resolve the correct image to execute your code on top the requested view, therefore you must always set the top level GitHub Action variable runs-on: ubuntu-latest.

run-cvmfs-osg-eic-shell's People

Contributors

wdconinc avatar

Watchers

Dmitry Kalinkin avatar Torre Wenaus avatar  avatar  avatar Sylvester Joosten avatar Sakib Rahman avatar Christopher Dilks avatar Holly Szumila avatar David Lawrence avatar Markus Diefenthaler avatar Chao Peng avatar  avatar

run-cvmfs-osg-eic-shell's Issues

Allow for running jobs without network access

Is your feature request related to a problem? Please describe.
Artifacts in epic builds are pulled from URLs when not in the container cache. For the large number of pipelines in epic, we want to disable repeated downloads from the original server, fail the job, and direct users to add the new artifacts to the container cache. This will significantly reduce GETs on the original locations.

This is also a good test case to enforce being able to run on network-isolated HPC nodes.

Describe the solution you'd like
The singularity call can disable network access based on an option to this action. All other network access to the previous steps in the job should be maintained to allow for checkout, cvmfs setup, etc.

Describe alternatives you've considered
I'm not sure of another way to achieve this.

Set up automatic release management into major release numbers

Is your feature request related to a problem? Please describe.
GitHub Actions are used in workflows by major version number only (e.g. actions/checkout@v3). We release here with a three-component semantic versioning number. This requires that we do the following upon a new release: delete the major version tag, create the new release, create a new major version tag. This is cumbersome and error-prone.

Describe the solution you'd like
One of the many GitHub Actions that facilitates this process would be useful here, e.g. https://github.com/marketplace/actions/build-and-tag. Many others are available

Describe alternatives you've considered
We can also just be clearer about the procedure we follow and make sure we keep doing that correctly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.