Table of contents:
- Introduction
- Prerequisites
- Installation
- Playground
- Troubleshooting
- Contributing
- Relevant links
- Credits and License
k8s-sysdig-adapter
is an implementation of the
Custom Metrics API using
Sysdig Monitor.
If you have a Kubernetes cluster and you are a Sysdig user, this adapter enables you to create horizontal pod autoscalers based on metrics provided by Sysdig's monitoring solution.
In the following example, we're creating an autoscaler based on the
net.http.request.count
metric. The autoscaler will adjust the number of pods
deployed for our service as the metric fluctuates over or below the threshold
(targetValue
).
---
kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
name: kuard
spec:
scaleTargetRef:
kind: Deployment
name: kuard
minReplicas: 3
maxReplicas: 10
metrics:
- type: Object
object:
target:
kind: Service
name: kuard
metricName: net.http.request.count
targetValue: 100
You're going to need:
- Kubernetes 1.8+
- Sysdig Monitor - see the installation instructions.
- Sysdig Monitor Access Key - which you've used during the installation of Sysdig Monitor.
- Sysdig Monitor API Token - see where to find it in these instructions. Do not confuse the API token with the agent access key, they're not the same! This is the API token that our metrics server is going to use when accessing the API.
The configuration files that you can find under the deploy directory are just for reference. Every deployment is unique so tweak them as needed. At the very least, you need to use your own access key and API token as follows:
- 01-sysdig-daemon-set.yml installs the Sysdig agent - edit it so it uses your own access key.
- 03-sysdig-metrics-server.yml installs
a
secret
in Kubernetes containing the Sysdig Monitor API token - edit it to use your own token.
Now we're ready to start! ๐
-
For the purpose of this example we're going to deploy kuard, a demo application found in the "Kubernetes Up and Running" book. This application is deployed with three replicas by default.
$ kubectl apply -f deploy/00-kuard.yml
Let's check that it's running:
$ kubectl get pods -l app=kuard -o wide NAME READY STATUS RESTARTS AGE IP NODE kuard-bcc7bf7df-clv2f 1/1 Running 0 1m 10.46.0.2 worker-node-2 kuard-bcc7bf7df-d9svn 1/1 Running 0 1m 10.40.0.2 worker-node-1 kuard-bcc7bf7df-zg8nc 1/1 Running 0 1m 10.46.0.3 worker-node-2
-
Install Sysdig Monitor if you haven't done it yet - they have great docs that you can use. In this example, we're going to install it using a
DaemonSet
object as follows:$ kubectl apply -f deploy/01-sysdig-daemon-set.yml
Don't forget to add your own access key to the file!
-
The following command is going to deploy a number of required objects like a custom namespace
custom-metrics
, required RBAC roles, permissions, bindings and the service object for our metrics server:$ kubectl apply -f deploy/02-sysdig-metrics-rbac.yml
-
Deploy the metrics server with:
$ kubectl apply -f deploy/03-sysdig-metrics-server.yml
Don't forget to add your own API token to the file!
It should be possible to retrieve the full list of metrics available using the following command:
$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq -r ".resources[].name"
-
Deploy our custom autoscaler that scales our service based on the
net.http.request.count
metric.$ kubectl apply -f deploy/04-kuard-hpa.yml
At this point you should be able to see the autoscaler in action. In the example, we set a threshold of 100 requests per minute. Let's generate some traffic with hey:
$ hey -c 5 -q 85 -z 24h http://10.103.86.213
Finally, use the following command to watch the autoscaler:
$ kubectl get hpa kuard-autoscaler -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
kuard-autoscaler Deployment/kuard 105763m/100 3 10 8 2d
You can find a playground based on Vagrant virtual machines under the playground directory in this repository. You can use it to demo this project or for development purposes.
minikube has not been tried yet! See issue #3 for more details.
If you encounter any problems that the documentation does not address, file an issue.
Thanks for taking the time to join our community and start contributing!
- Please familiarize yourself with the Code of Conduct before contributing.
- See CONTRIBUTING.md for information about setting up your environment, the workflow that we expect, and instructions on the developer certificate of origin that we require.
- Check out the issues and our roadmap.
From the Kubernetes project:
- Types in Custom Metrics API
- Library for writing a Custom Metrics API server
- Library for writing a Kubernetes-style API Server
- Documentation: Horizontal Pod Autoscaling
Other links:
- Sysdig's blog post about HPA
- Kubernetes Prometheus Adapter
- kubeadm workshop
- Basic Custom Metrics API server using minikube
This project wouldn't be possible without the great job done by others. See the links above for some of the things I've been using in different ways.
The tagging policy and the contributing guide in this project is based on Contour's.
This project is open source and it uses a permissive license.