Runtime Specific Metrics via `ServiceMonitor` (with qpext) about kserve HOT 7 CLOSED

@scottclusk The port is exposed with name aggr-metric on pod. https://github.com/kserve/kserve/pull/3291/files

from kserve.

You can use PodMonitor instead of using ServiceMonitor. prometheus-operator/prometheus-operator#3119

from kserve.

@sivanantha321 thanks. This is the approach i needed up taking and it worked well

from kserve.

Hello @robertgshaw2-neuralmagic any chance you elaborate on the fix please? I'm having the same issue, and it seems to me that the podMonitor also requires a named port. targetport is deprecated and port requires a number no? thanks :)

from kserve.

Hello @robertgshaw2-neuralmagic any chance you elaborate on the fix please? I'm having the same issue, and it seems to me that the podMonitor also requires a named port. targetport is deprecated and port requires a number no? thanks :)

WIP example -> https://github.com/robertgshaw2-neuralmagic/vllm-k8s/tree/main/vllm-kserve

I ended up using the following:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: llama-3-monitor
spec:
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      serving.kserve.io/inferenceservice: llama-3
  podMetricsEndpoints:
  - port: user-port
    interval: 5s

The user-port is 8000 ... the InferenceService seems to expose this

from kserve.

Hello @robertgshaw2-neuralmagic any chance you elaborate on the fix please? I'm having the same issue, and it seems to me that the podMonitor also requires a named port. targetport is deprecated and port requires a number no? thanks :)

WIP example -> https://github.com/robertgshaw2-neuralmagic/vllm-k8s/tree/main/vllm-kserve

I ended up using the following:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: llama-3-monitor
spec:
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      serving.kserve.io/inferenceservice: llama-3
  podMetricsEndpoints:
  - port: user-port
    interval: 5s

The user-port is 8000 ... the InferenceService seems to expose this

How does it find your aggregate metrics though, aren't those being exposed on 9088?

from kserve.

It doesn't aggregate.

But you could then open up a Servicemonitor that hits the 9091 if you want to get the queue_proxy metrics

from kserve.