Armada is an experimental application to achieve high throughput of run-to-completion jobs on multiple Kubernetes clusters.

It stores queues for users/projects with pod specifications and creates these pods once there is available resource in one of the connected Kubernetes clusters.

• Design Documentation
• Development Guide
• User Guide
• Installation in Production

• Armada maintains fair resource share over time (inspired by HTCondor priority)
• It can handle large amounts of queued jobs (million+)
• It allows adding and removing clusters from the system without disruption
• By utilizing multiple Kubernetes clusters the system can scale beyond the limits of a single Kubernetes cluster

Queue: Represent user or project, used to maintain fair share over time, has priority factor

Job: Unit of work to be run (described as Kubernetes PodSpec)

Job Set: Group of related jobs, api allows observing progress of job set together

Prerequisites:

• Git
• Go 1.12+
• Docker installed. Ensure the current user has permission to run the docker command without sudo.
• 3GB Disk space

1. Clone repository & build

git clone https://github.com/G-Research/armada.git
cd armada
make build

2. Get Kind (Installation help here)

Kind is Kubernetes in Docker. It allows us to easily run a local Kubernetes cluster using Docker.

GO111MODULE="on" go get sigs.k8s.io/[email protected]

3. Create 2 Kind clusters

As this step is using Docker, it will require root to run

kind create cluster --name demoA --config ./example/kind-config.yaml
kind create cluster --name demoB --config ./example/kind-config.yaml 

4. Start Redis

docker run -d -p 6379:6379 redis

5. Start server in one terminal

./bin/server

6. Start executor for demoA cluster in a new terminal

KUBECONFIG=$(kind get kubeconfig-path --name="demoA") ARMADA_APPLICATION_CLUSTERID=demoA ARMADA_METRICSPORT=9001 ./bin/executor

7. Start executor for demoB cluster in a new terminal

KUBECONFIG=$(kind get kubeconfig-path --name="demoB") ARMADA_APPLICATION_CLUSTERID=demoB ARMADA_METRICSPORT=9002 ./bin/executor

8. Create queue, submit jobs and watch progress

./bin/armadactl create-queue test --priorityFactor 1
./bin/armadactl submit ./example/jobs.yaml
./bin/armadactl watch job-set-1