Code Monkey home page Code Monkey logo

workflows's Introduction

workflows

A set of example Argo workflows.

Provisioning

To provision Argo, we use the Argo Workflows ansible role and the provisioning playbook.

You will be asked a namespace to with the Argo should be provisioned and a password to the vault. Run:

ansible-playbook playbooks/provision.yml --ask-vault-pass

Submitting workflows

Using the Argo CLI

See How to write Argo Workflows for the getting started.

To submit a more complicated workflow (i.e., the inspection), first make sure that all the templates have been created.

kubectl create --namespace ${ARGO_NAMESPACE} -f examples/templates

And now to submit the inspection workflow using the parameters file:

argo submit --watch examples/inspection.yaml \
    --name inspection-131719234 \
    -p batch-size=${BATCH_SIZE} \
    -p parallelism=${PARALLELISM} \
    -p thoth-infra-namespace=${ARGO_NAMESPACE} \
    -f examples/parameters.json

Submitting workflows via Python

See examples notebooks or check thoth.common.workflows.

Submitting a Solver Workflows

Solver workflows require a few WorkflowTemplate: oc apply -f argo/templates/ and if this prerequisite is fulfilled, a Workflow can be created using:

argo submit --watch argo/solve_and_sync.yaml \
    --parameter THOTH_SOLVER_NAME=rhel-8.0-py36 \
    --parameter THOTH_SOLVER_JOB_ID=20191126115542 \
    --parameter THOTH_SOLVER_PACKAGES="tensorflow==2.0.0"

workflows's People

Contributors

goern avatar harshad16 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

workflows's Issues

[5pt] create Argo workflow to generate SI observation

As Thoth,
I want to run an Argo Workflow
so that a bandit-based SI report is generated,
and so that the report is stored to S3
and so that the report is synced to the graph

Story-Points: 5

Acceptance Criteria

  • run workflow from commandline
  • workflow will:
    • download index_package_version to a tmp dir
    • compute SI on index_package_version -> S3
    • aggregate it into one single score -> S3
    • sync to graph

create workflow to analyse missing packages

Based on an advise log, we can create a list of python packages that are missing in our knowledge graph. This list of packages should be added by submitting them for solving and analysis.
The workflow could use the advise log as an input, and could be triggered by all advise runs.

zuul integration

please integrate with zuul and run at least the yaml linter in check and gate pipelines.

Zuul is not merging...

Describe the bug
zuul is not merging, even if the approved label and status checks are ok.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.