promhippie / prometheus-vcd-sd Goto Github PK
View Code? Open in Web Editor NEWPrometheus Service Discovery for VMWare vCloud Director
Home Page: https://promhippie.github.io/prometheus-vcd-sd/
License: Apache License 2.0
Prometheus Service Discovery for VMWare vCloud Director
Home Page: https://promhippie.github.io/prometheus-vcd-sd/
License: Apache License 2.0
The HTTP SD within Prometheus accepts only application/json
as a content type, so we need to drop the utf-8 encoding part from the content type.
Hi, Thank you for the great work, this is a useful tool.
We would like to use fixed tags in our deployment, but the 0.4.0 doesn't include #142 which affects our systems.
Can you create a new release tag?
We got to drop the darwin/386 builds, they are not supported by the used Go version anymore. Soonish we should also add arm64 builds to stay compatible with current macOS hardware.
It would be nice if a change to the configuration automatically reloads the service.
Since Prometheus implements a HTTP SD with the new version 2.28 we should provide an endpoint for this implementation as an alternative to the regular file handler.
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: .github/renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid configuration option: dockerfile.enableMajor
Hi,
We found you project very interesting but is it possible to add the esx host information on service discovery ?
The json output file doesn't include this interesting information and we would like to supervise when a node move to an other esx host.
Thanks a lot
To have a unified project structure accross the whole promhippie GitHub organization all projects should impelent a similar structure.
Target labels should contain the VM ID as provided by VCD.
E.g.: "__meta_vcd_id": "urn:vcloud:vm:807b799e-c72f-4592-9acd-ccefefe92720"
Hi, while using VCD-SD exporter, find this bug.
Discover service from prometheus-vcd-sd
{
"targets": [
"10.12.146.50"
],
"labels": {
"__address__": "10.12.146.50",
"__meta_vcd_id": "urn:vcloud:vm:02217ed9-ea98-495e-a577-d1a517017611",
"__meta_vcd_name": "k8sw-dev-cubone",
"__meta_vcd_network_k8s_net": "10.12.146.50",
"__meta_vcd_num_cores_per_socket": "2",
"__meta_vcd_num_cpus": "8",
"__meta_vcd_org": "VDC",
"__meta_vcd_os_type": "ubuntu64Guest",
"__meta_vcd_project": "default",
"__meta_vcd_status": "4",
"__meta_vcd_storage_profile": "NonSSD",
"__meta_vcd_vdc": "VDC"
}
},
{
"targets": [
"10.12.147.4"
],
"labels": {
"__address__": "10.12.147.4",
"__meta_vcd_id": "urn:vcloud:vm:12533e37-9ba1-4cd0-8919-d0360b6acdcf",
"__meta_vcd_metadata_vmware.cloud.director.ui.badges.blue": "false",
"__meta_vcd_metadata_vmware.cloud.director.ui.badges.green": "false",
"__meta_vcd_metadata_vmware.cloud.director.ui.badges.magenta": "false",
"__meta_vcd_metadata_vmware.cloud.director.ui.badges.orange": "false",
"__meta_vcd_metadata_vmware.cloud.director.ui.badges.red": "false",
"__meta_vcd_name": "jump-host",
"__meta_vcd_network_srv_net": "10.12.147.4",
"__meta_vcd_num_cores_per_socket": "1",
"__meta_vcd_num_cpus": "2",
"__meta_vcd_org": "VDC",
"__meta_vcd_os_type": "ubuntu64Guest",
"__meta_vcd_project": "default",
"__meta_vcd_status": "4",
"__meta_vcd_storage_profile": "NonSSD",
"__meta_vcd_vdc": "VDC"
}
},
After this in log Prometheus contain error
prometheus | ts=2022-02-15T09:44:43.523Z caller=refresh.go:79 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="\"__meta_vcd_metadata_vmware.cloud.director.ui.badges.green\" is not a valid label name"
docker-compose.yml
version: '2'
volumes:
prometheus:
services:
prometheus:
image: prom/prometheus
container_name: prometheus
restart: always
ports:
- 9090:9090
volumes:
- prometheus:/prometheus
- ./prometheus.yml:/etc/prometheus/prometheus.yml
- ./service-discovery:/etc/sd
vcd-exporter:
image: promhippie/prometheus-vcd-sd:latest
container_name: vcd-sd
restart: always
ports:
- 9000:9000
environment:
- PROMETHEUS_VCD_LOG_PRETTY=true
- PROMETHEUS_VCD_OUTPUT_ENGINE=http
- PROMETHEUS_VCD_OUTPUT_FILE=/etc/sd/vcd.json
- PROMETHEUS_VCD_URL=API_URL
- PROMETHEUS_VCD_USERNAME=USER
- PROMETHEUS_VCD_PASSWORD=PASS
- PROMETHEUS_VCD_ORG=VDC
- PROMETHEUS_VCD_VDC=VDC
volumes:
- ./service-discovery:/etc/sd
prometheus.yml
global:
scrape_interval: 1m
scrape_timeout: 10s
evaluation_interval: 1m
scrape_configs:
- job_name: node
http_sd_configs:
- url: http://vcd-sd:9000/sd
relabel_configs:
- source_labels: [__meta_vcd_network_internal]
replacement: "${1}:9100"
target_label: __address__
- source_labels: [__meta_vcd_name]
target_label: instance
docker version: 20.10.12
docker compose version: v2.2.3
OS: Debian GNU/Linux 11 (bullseye)
While the service is running fine it could happen after some amount of time that the authentication gets invalidaded, that would require to restart the service as it doesn't try to re-authenticate. Maybe it would make sense to authenticate on every loop and disconnect at the end of it.
Instead of adding development tooling to the regular go.mod file we should use bingo to version used tools like calens, golint, reflex and staticcheck to make sure we always use versions known to work properly.
For Prometheus there is a new toolkit available which enables a web config to secure exporters (or HTTP service discoveries) be integrating certs and also authentication. This should be implemented. You can find it at https://github.com/prometheus/exporter-toolkit.
If a vApp does not provide any VM currently it fails to process any service discovery as it panics. We got to check if a vApp provides any VM before looping through it.
Just prepared an initial basic version which could be released to the public.
Hi :)
I'm getting this error :
go:337 level=error component="discovery manager scrape" discovery=file msg="Error reading file" path=/etc/sd/vcd-vdcplus.json err="\"__meta_vcd_network_network name\" is not a valid label name"
It seems that a space in any label name is not good for Prometheus.
I'm not confident in developing in Go but if you don't have time I can take a look at it :)
Every network attached to the VMs gets transformed into a label, but currently there is no normalization of the network names. A network name like FOO-bar-1000
should be translated to __meta_vcd_network_foo_bar_1000
, otherwise it's an invalid label.
Hi,
Thanks for this wonderful project not many project at github that can combine vcloud director metrics into prometheus.
But i still wondering how to use it? since, my prometheus can't read any information from output.log.
It can only read data like this. There's no info in output.log such as __meta_vcd_metadata_apps, __meta_vcd_num_cores_per_socket that listed in vcd-sd result page (http://localhost:9000/metrics)
are there any possibilities to read whatever in output.log said into Prometheus, coz output.log data is accurate but somehow can't be injected into vcd-sd metrics data (http://localhost:9000/metrics)
Thanks
We need to update the used Go version and on the same run we should also update all dependencies to the latest version to keep everything properly up to date.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
This repository currently has no open or pending branches.
docker/Dockerfile.linux.386
i386/alpine 3.20@sha256:fa66aa594ffa884dff44f4a97821756545834505df611c375a30c45926402f89
docker/Dockerfile.linux.amd64
alpine 3.20@sha256:eddacbc7e24bf8799a4ed3cdcfa50d4b88a323695ad80f317b6629883b2c2a78
docker/Dockerfile.linux.arm
arm32v6/alpine 3.20@sha256:5c7e326e3c8a8c51654a6c5d94dac98d7f6fc4b2a762d86aaf67b7e76a6aee46
docker/Dockerfile.linux.arm64
arm64v8/alpine 3.20@sha256:24ba417e25e780ff13c888ccb1badec5b027944666ff695681909bafe09a3944
.github/workflows/automerge.yml
tibdex/github-app-token v2
dependabot/fetch-metadata v2
.github/workflows/binaries.yml
actions/checkout v4
actions/setup-go v5
actionhippie/gpgsign v1
actionhippie/calens v1
ncipollo/release-action v1
.github/workflows/changes.yml
actions/checkout v4
actions/checkout v4
actions/setup-go v5
EndBug/add-and-commit v9
actions/checkout v4
actions/checkout v4
actions/setup-go v5
EndBug/add-and-commit v9
actions/checkout v4
actions/checkout v4
actions/setup-go v5
EndBug/add-and-commit v9
.github/workflows/docker.yml
actions/checkout v4
actions/setup-go v5
docker/metadata-action v5
docker/setup-qemu-action v3
docker/setup-buildx-action v3
docker/login-action v3
docker/login-action v3
docker/login-action v3
docker/build-push-action v6
actions/checkout v4
docker/metadata-action v5
actionhippie/manifest v1
docker/metadata-action v5
actionhippie/manifest v1
docker/metadata-action v5
actionhippie/manifest v1
actions/checkout v4
actionhippie/pushrm v1
actionhippie/pushrm v1
.github/workflows/docs.yml
actions/checkout v4
peaceiris/actions-hugo v3
peaceiris/actions-gh-pages v4
.github/workflows/flake.yml
tibdex/github-app-token v2
actions/checkout v4
cachix/install-nix-action v27
DeterminateSystems/update-flake-lock v23
.github/workflows/general.yml
actions/checkout v4
actions/setup-go v5
codacy/codacy-coverage-reporter-action v1
.github/workflows/kustomize.yml
actions/checkout v4
actionhippie/kustomize v2
go.mod
go 1.22
go 1.22.6
github.com/go-chi/chi/v5 v5.1.0
github.com/go-kit/log v0.2.1
github.com/joho/godotenv v1.5.1
github.com/oklog/run v1.1.0
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/common v0.55.0
github.com/prometheus/exporter-toolkit v0.11.0
github.com/stretchr/testify v1.9.0
github.com/urfave/cli/v2 v2.27.3
github.com/vmware/go-vcloud-director/v2 v2.25.0
gopkg.in/yaml.v3 v3.0.1
deploy/kubernetes/kustomization.yml
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.