Code Monkey home page Code Monkey logo

prometurbo's Introduction

Prometurbo

Get metrics from Prometheus for applications, and expose these applications and metrics in JSON format via REST API. The data ingestion framework probe ( i.e. DIF Probe) will access the REST API, convert the JSON output to Turbonomic DTO to be consumed by Turbonomic server. This enables Turbonomic to collect and analyze Prometheus metrics and make intelligent decisions about application scaling, placement and optimization.

appmetric

To configure the Prometheus server and map query results into applications and metrics, you need to create the following custom resources in the Kubernetes cluster (from Prometurbo 8.8.4):

  • PrometheusQueryMapping: allows users to define mappings between Turbonomic entities (such as ApplicationComponents, Services, or VirtualMachines) and Prometheus metrics exposed by different prometheus exporters.
  • PrometheusServerConfig: specifies the address of the Prometheus server, as well as optional label selectors to filter out PrometheusQueryMapping resources applicable to that server. This allows users to configure multiple Prometheus servers and use different mappings for each server.

Custom resource definitions for the above two resources must be installed first in the Kubernetes cluster. Get them here.

Sample custom resource instances can be found here.

Output of Prometurbo: Applications with their metrics

The application metrics are served via REST API at endpoint /metrics. The output JSON format is defined at turbo-go-sdk:

type Topology struct {
	Version    string       `json:"version"`
	Updatetime int64        `json:"updateTime"`
	Scope      string       `json:"scope"`
	Source     string       `json:"source"`
	Entities   []*DIFEntity `json:"topology"`
}

type DIFEntity struct {
	UID                 string                     `json:"uniqueId"`
	Type                string                     `json:"type"`
	Name                string                     `json:"name"`
	HostedOn            *DIFHostedOn               `json:"hostedOn"`
	MatchingIdentifiers *DIFMatchingIdentifiers    `json:"matchIdentifiers"`
	PartOf              []*DIFPartOf               `json:"partOf"`
	Metrics             map[string][]*DIFMetricVal `json:"metrics"`
	partOfSet           set.Set
	hostTypeSet         set.Set
}

Deploy

Follow the deployment instructions at here to deploy Prometurbo and DIFProbe container in the same Pod.

prometurbo's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prometurbo's Issues

Proxy entities should be removed for deleted pods

Prometurbo will created proxy entities for those deleted pods as Prometheus server will still have the metrics available. Prometurbo currently has no way to figure out if pods are deleted.

At server side, the proxy entities of the deleted pods will not be stitched with kubeturbo. It's not desired to show them in UI. So, need to investigate how to remove the proxy entities in SDK framework if they are not stitched at server side.

Support generic addon to allow specify metric query in configuration

Currently, when added one addon to support a new exporter, we need to write code for the exporter.

After looking into some examples as Cassandra and XL server, it seems that we can have one generic addon that can allow users to specify the queries to get the metrics and the mapping from the metrics to the corresponding commodities. For example, user can simply add following to a configuration file:

promethuest-query-1 RESPONSE_TIME
promethuest-query-2 TRANSACTION

and prometurbo will take the settings and query the promethues server for the metrics and generate response time and transaction commodities as in our current prometurbo mechanism.

The above is a simplified version but as discussed with the team, this generic addon should be doable and shouldn't be too complicated. Need further investigation and think an optimal way for the framework.

Stitching doesn't work for a service with more than one pod

The vApp (service) commodities of transaction and response time doesn't get updated if the service has more than one ContainerPod instances associated.

Some suspect is on stitching at Server side. Need further investigation about the root cause.

Unable to customise the configmap with the operator deployment.

The operator is looking for global values for prometheusServers
https://github.com/turbonomic/prometurbo/blob/master/deploy/prometurbo/templates/configmap-prometurbo.yaml

  name: prometurbo-config-{{ .Release.Name }}
data:
  prometheus.config: |-
    # Prometheus servers and metrics to scrape
    servers:
{{- if .Values.prometheusServers }}
{{ tpl .Values.prometheusServers . | indent 6 }}
{{- end }}

but there is no value defined for it in the config
deploy/prometurbo-operator/config/crd/bases/charts.helm.k8s.io_prometurboes.yaml

The outcome of this is a default configmap with
https://github.com/turbonomic/prometurbo/blob/master/deploy/prometurbo/values.yaml

prometheusServers: |
  server1:
    # The URL of the prometheus server
    url: http://Prometheus_Server_URL
    # The list of supported exporters for the above prometheus server
    exporters:
      - cassandra
      - istio
      - jmx-tomcat
      - node
      - redis
      - webdriver

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.