Code Monkey home page Code Monkey logo

benchmarks's Introduction

Cloud Costs Benchmarks

This repository collects the RNA-Seq, Variant calling, and Single Cell benchmarks that are scattered amongst several other repositories including, but not limited to:

  1. Resource Prediction Experiments
  2. Biology of Genomes
  3. Experiments

Experiments

Each of the rnaseq, variant, and single-cell directories contain the benchmark, experiment, and TPV configuration files required to run the benchmark experiments.

Results Tables

  1. RNA-Seq
  2. Ice Lake

Files

costs.sh
Environment variable definitions used when creating a GKE cluster. See Variables below for a description of the variables used.
cascade-lake.sh
Variable definitions used for the clusters using Intel Cascade Lake cpus.
ice-lake.sh
Variable definitions used for the clusters using Intel Ice Lake cpus.
anvil
is a Bash script used to manage the GKE clusters used to run the experiments. Run anvil --help for full usage information.

Ensuring the CPU Type

Google supports specifying a --min-cpu-platform, but that is a minimum and we may get a later CPU family if the minium requested family is not available. Therefore we need to ensure the CPUs used in the cluster are actually the type requested.

# SSH into the node
gcloud compute ssh --zone <ZONE> <instance name>

# Install GCC
sudo apt update
sudo apt install build-essential

# Get the CPU family name
gcc -march=native -Q --help=target | grep -march

Create a cluster

anvil --settings settings.sh cluster disks galaxy

Destroying a cluster

anvil --settings settings.sh cleanup delete-disks

If you do not delete the persistent disks they can be re-attached to a new cluster as long as the same cluster --prefix is used. The easiest way to achieve this is to use a settings file that contains definitions for the common variable definitions.

anvil --settings settings.sh cluster disks galaxy
anvil --settings settings.sh cleanup 
anvil --settings settings.sh cluster galaxy # Will re-use the persistent disks created in step 1

Variables

The following environment variables can be set that will be used by the anvil Bash script.

MACHINE
Machine type for cluster nodes, e.g. n2-standard-32.

PREFIX
A prefix string used to when creating resources.

NAMESPACE
Kubernetes/Helm namespace used when installing Galaxy. Defaults to galaxy.

IMAGE
Docker image used to install Galaxy. Defaults to quay.io/galaxyproject/galaxy-anvil.

TAG
Image tag of the Docker image used to install Galaxy. Defaults to 23.1.

CHART
Helm chart used to install Galaxy. Defaults to anvil/galaxykubeman.

DISK
Size of the persistent disk created. In GB. Defaults to 768.

benchmarks's People

Contributors

ksuderman avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.