Comments (10)
I managed to get the example deployment to work.
Had to make a couple of changes as mentioned below:
- As mentioned in the example deployment, step 2 and 3 need to be interchanged, we need to create the namespace followed by the secret which specifies the namespace.
kubectl create namespace custom-metrics
export PURPOSE=serving
openssl req -x509 -sha256 -new -nodes -days 365 -newkey rsa:2048 -keyout ${PURPOSE}-ca.key -out ${PURPOSE}-ca.crt -subj "/CN=ca"
echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","'${PURPOSE}'"]}}}' > "${PURPOSE}-ca-config.json"
kubectl -n custom-metrics create secret tls cm-adapter-serving-certs --cert=./serving-ca.crt
--key=./serving-ca.key
- Needed to modify custom-metrics-apiserver-deployment.yaml changing the values for --tls-cert-file and --tls-private-key-file arguements to the one's mentioned while describing the secret as shown below.
kubectl describe secret cm-adapter-serving-certs -n custom-metrics
Name: cm-adapter-serving-certs
Namespace: custom-metrics
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.key: 1704 bytes
tls.crt: 1074 bytes
So in my case i changed it from serving.crt and serving.key to tls.cert and tls.key
from prometheus-adapter.
I also faced the issue here:
https://github.com/DirectXMan12/k8s-prometheus-adapter/blame/master/docs/walkthrough.md#L219
$ kubectl create clusterrole resource-lister --verb=list --resource="*"
the server doesn't have a resource type "*"
But if I do kubectl create -f
it works with the following
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: resource-lister
rules:
- apiGroups:
- ""
resources:
- '*'
verbs:
- list```
from prometheus-adapter.
I think there was actually a bug in kubectl create clusterrole
that prevented --resource=*
from working. If you choose a specific resource, and then edit the YAML afterwards to *
, it should work.
were you following the tutorial where it said to create the a configmap with your prometheus configuration?
from prometheus-adapter.
Thanks, il do the change to the yaml.
Yes i was following the walkthrough and i did create the configmap with the same prom-cfg.yaml that was mentioned above(https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/docs/walkthrough.md) as it said it might just be the above.
I also tried the example deployment(https://github.com/DirectXMan12/k8s-prometheus-adapter/tree/master/deploy). Modified as necessary to point to my prometheus server. And executed the below commands:
export PURPOSE=serving
openssl req -x509 -sha256 -new -nodes -days 365 -newkey rsa:2048 -keyout ${PURPOSE}-ca.key -out ${PURPOSE}-ca.crt -subj "/CN=ca"
echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","'${PURPOSE}'"]}}}' > "${PURPOSE}-ca-config.json"
kubectl create secret tls cm-adapter-serving-certs --cert=./serving-ca.crt --key=./serving-ca.key
kubectl create namespace custom-metrics
cd deploy
kubectl create -f manifests/
The pod shows up with status
NAMESPACE NAME READY STATUS RESTARTS AGE
custom-metrics custom-metrics-apiserver-66db884599-2xp4c 0/1 ContainerCreating 0 49s
On doing a describe, i see a volume mount error as shown below:
# kubectl describe pod custom-metrics-apiserver -n custom-metrics
Name: custom-metrics-apiserver-66db884599-2xp4c
Namespace: custom-metrics
Node: XXXXXXXXXXXX
Start Time: Tue, 24 Apr 2018 00:53:49 -0400
Labels: app=custom-metrics-apiserver
pod-template-hash=2286440155
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/custom-metrics-apiserver-66db884599
Containers:
custom-metrics-apiserver:
Container ID:
Image: directxman12/k8s-prometheus-adapter
Image ID:
Port: 6443/TCP
Args:
/adapter
--secure-port=6443
--tls-cert-file=/var/run/serving-cert/serving.crt
--tls-private-key-file=/var/run/serving-cert/serving.key
--logtostderr=true
--prometheus-url=http://localhost:9090/
--metrics-relist-interval=30s
--rate-interval=5m
--v=10
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from custom-metrics-apiserver-token-pppk8 (ro)
/var/run/serving-cert from volume-serving-cert (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
volume-serving-cert:
Type: Secret (a volume populated by a Secret)
SecretName: cm-adapter-serving-certs
Optional: false
custom-metrics-apiserver-token-pppk8:
Type: Secret (a volume populated by a Secret)
SecretName: custom-metrics-apiserver-token-pppk8
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 1m default-scheduler Successfully assigned custom-metrics-apiserver-66db884599-2xp4c to XXXXXXXX
Normal SuccessfulMountVolume 1m kubelet, XXXXXXXX MountVolume.SetUp succeeded for volume "custom-metrics-apiserver-token-pppk8"
Warning FailedMount 25s (x8 over 1m) kubelet, XXXXXXXX MountVolume.SetUp failed for volume "volume-serving-cert" : secrets "cm-adapter-serving-certs" not found
I dont really know where have i gone wrong. Any help would be appreciated.
from prometheus-adapter.
Thanks @zioproto .
Also I noticed that the prom-adapter.deployment.yaml file needs a change as the volumeMounts and volumes need to have the same name.
volumeMounts:
# you'll mount your ConfigMap volume at /etc/prometheus
- mountPath: /etc/prometheus
name: prom-config
volumes:
# make your configmap available as a volume, so that you can
# mount in the Prometheus config from earlier
- name: config-volume
configMap:
name: prometheus
In volumeMounts it mentions prom-config, whereas the volume mentioned has name config-volume.
from prometheus-adapter.
closing, since it seems like we've fixed these issues
from prometheus-adapter.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale
.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close
.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale
from prometheus-adapter.
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten
.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close
.
Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
from prometheus-adapter.
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen
.
Mark the issue as fresh with /remove-lifecycle rotten
.
Send feedback to sig-contributor-experience at kubernetes/community.
/close
from prometheus-adapter.
@fejta-bot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity.
Reopen the issue with/reopen
.
Mark the issue as fresh with/remove-lifecycle rotten
.Send feedback to sig-contributor-experience at kubernetes/community.
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
from prometheus-adapter.
Related Issues (20)
- how to aggregate query with multiple metrics ? HOT 4
- how to ignore parameters from hpa request ? HOT 3
- is it possible to use external prometheus server on ec2 server? HOT 1
- Scaling model for prometheus-adapter after disabling metrics-server HOT 1
- How do i get the v0.11.2 docker image? HOT 2
- Documentation on how to directly query the adapter HOT 2
- Panic slice out of bands when querying namespace metric HOT 1
- Monitoring of prometheus-adapter metrics HOT 3
- Error while disabling TLS in server, setting secure-port = 0 throws error HOT 3
- Improve security hygiene and documentation HOT 3
- Unable to see Node Metrics - Error Metrics Missing CPU for node "XXX", skipping HOT 3
- Kubectl --raw reporting an unknown metric even though it shows up in the list of known metrics HOT 1
- Issue fetching external metric HOT 1
- Failed to get pods metric value: unable to get metric DCGM_FI_DEV_FB_USED_AVG: no metrics returned from custom metrics API HOT 5
- Prometheus adapter support for cross cluster autoscaling HOT 3
- Failed to pull image "quay.io/coreos/k8s-prometheus-adapter-amd64:v0.4.1" HOT 3
- Adding custom CRD for Prometheus Adapter rules HOT 1
- Correct Configuration Fails to Provide Expected Custom Metrics in EKS HOT 2
- Why the custom-metric-api yaml file lost after release-0.11? HOT 4
- external.metrics api does not goes up. only custom.metrics api. ='( HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prometheus-adapter.