k-phoen / dark Goto Github PK

(grafana) Dashboards As Resources in Kubernetes

License: MIT License

Go 95.59% Makefile 4.41%

grafana grafana-dashboard kubernetes kubernetes-crd dashboard-configuration kubernetes-controller yaml dashboard dashboards hacktoberfest

dark's Introduction

DARK

Dashboards As Resources in Kubernetes.

DARK provides a way to define and deploy Grafana dashboards via Kubernetes, next to the services they monitor.

If you are looking for a way to version your dashboards and deploy them across all environments, like you would do with your services, then this project is meant for you.

In addition to dashboards, DARK can also manage:

data sources
API keys
alert manager configuration (alert routing, notification policies, ...)

Design goals

full description of dashboards via YAML
integration with kubectl
seamless integration with Grafana
delegate YAML decoding and dashboard generation to Grabana

Non-goals

creation and operation of Grafana instances themselves. For that, check out grafana-operator/grafana-operator

Example dashboard

apiVersion: k8s.kevingomez.fr/v1
kind: GrafanaDashboard
metadata:
  name: example-dashboard
  namespace: monitoring
  annotations:
    dark/folder: "Awesome folder"
spec:
  title: Awesome dashboard
  shared_crosshair: true
  tags: [generated, yaml]
  auto_refresh: 10s

  variables:
    - interval:
        name: interval
        label: interval
        default: 1m
        values: [30s, 1m, 5m, 10m, 30m, 1h, 6h, 12h]

  rows:
    - name: Prometheus
      panels:
        - timeseries:
            title: HTTP Rate
            datasource: prometheus-default
            targets:
              - prometheus:
                  query: "rate(promhttp_metric_handler_requests_total[$interval])"
                  legend: "{{ handler }} - {{ code }}"

        - graph:
            title: Heap allocations
            datasource: prometheus-default
            targets:
              - prometheus:
                  query: "go_memstats_heap_alloc_bytes"
                  legend: "{{ job }}"

More examples can be found in the examples/ folder

Installation & usage

Check out the documentation to dig deeper into how to set up and use DARK.

Adopters

Companies using DARK.

They talk about DARK

License

This library is under the MIT license.

dark's People

Contributors

Stargazers

Watchers

dark's Issues

Comparison to Grafana Operator?

I like what I read about this project in a recent article.

As I dig in deeper, I realize that Grafana Operator also has CRDs for GrafanaDashboard, GrafanaDatasource, etc.

It would be useful to me if you added a comparison between the CRDs in this project and those in Grafana Operator.

Vulerabilites

Trivy scanner shows there are 2 vulnerabilities with possible fix in the latest docker image (kphoen/dark:v0.5.7).

How to reproduce:

Install trivy (https://github.com/aquasecurity/trivy)
Scan: trivy kphoen/dark:v0.5.7

Output:

kphoen/dark:v0.5.7 (alpine 3.14.1)
==================================
Total: 4 (UNKNOWN: 0, LOW: 0, MEDIUM: 2, HIGH: 2, CRITICAL: 0)

+--------------+------------------+----------+-------------------+---------------+--------------------------------------+
|   LIBRARY    | VULNERABILITY ID | SEVERITY | INSTALLED VERSION | FIXED VERSION |                TITLE                 |
+--------------+------------------+----------+-------------------+---------------+--------------------------------------+
| libcrypto1.1 | CVE-2021-3711    | HIGH     | 1.1.1k-r0         | 1.1.1l-r0     | openssl: SM2 Decryption              |
|              |                  |          |                   |               | Buffer Overflow                      |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3711 |
+              +------------------+----------+                   +               +--------------------------------------+
|              | CVE-2021-3712    | MEDIUM   |                   |               | openssl: Read buffer overruns        |
|              |                  |          |                   |               | processing ASN.1 strings             |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3712 |
+--------------+------------------+----------+                   +               +--------------------------------------+
| libssl1.1    | CVE-2021-3711    | HIGH     |                   |               | openssl: SM2 Decryption              |
|              |                  |          |                   |               | Buffer Overflow                      |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3711 |
+              +------------------+----------+                   +               +--------------------------------------+
|              | CVE-2021-3712    | MEDIUM   |                   |               | openssl: Read buffer overruns        |
|              |                  |          |                   |               | processing ASN.1 strings             |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3712 |
+--------------+------------------+----------+-------------------+---------------+--------------------------------------+

Info

Grafana version: v8.4.3
Dark: v0.9.2

First of all, I want to say this is an awesome tool! Out of everything I've looked at DARK seems to work the best for supporting the idea of Dashboards as Code in an easy-to-read YAML format.

Grafana is a very feature-rich product so it comes with no surprise that DARK and Grabana don't currently handle every option. Here is a list of settings and issues I have come across while trying to port some pre-existing dashboards over to DARK.

The converter tool does not handle converting most of my pre-existing dashboards. I'm guessing my dashboards contain settings which DARK is unable to convert.

Example:

❯ docker run --rm -it -u $(id -u):$(id -g) -v $(pwd):/workspace kphoen/dark-converter:latest convert-yaml \
        -i dashboard.json \
        -o converted-dashboard.yaml

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
2022-04-12T22:32:34Z	ERROR	converter/json.go:113	could not unmarshall dashboard	{"error": "json: cannot unmarshal object into Go struct field Board.panels of type string"}
2022-04-12T22:32:34Z	ERROR	converter/json.go:55	could parse input	{"error": "json: cannot unmarshal object into Go struct field Board.panels of type string"}
2022-04-12T22:32:34Z	FATAL	cmd/toyaml.go:30	Could not convert dashboard	{"error": "json: cannot unmarshal object into Go struct field Board.panels of type string"}

When trying to manually port dashboards from grafana over to DARK I have run into some missing features:

Missing Features

I tried to look through all the docs and code. It's possible I simply missed the correct fields.

Alerts

Example:

apiVersion: k8s.kevingomez.fr/v1
kind: GrafanaDashboard
metadata:
  name: sample-dashboard
folder: Sample
spec:
  title: Sample Dashboard
  editable: true
  shared_crosshair: true
  tags: [dark, generated]
  auto_refresh: 5s
  time: [now-6h, now]
  timezone: utc

  variables:
    - datasource:
        name: Source
        label: Prometheus
        type: prometheus
  rows:
    - name: SLIs
      panels:
        - timeseries:
            title: Uptime
            description: Synthetic probe success rate for ArgoCD
            datasource: $Source
            span: 4
            height: 350px
            targets:
              - prometheus:
                  query: probe_success{instance="https://argocd.localhost", environment="mgmt"}
                  legend: Uptime
                  ref: A
            visualization:
              gradient_mode: opacity
              fill_opacity: 10
              tooltip:
                "all_series"
            axis:
              unit: percentunit
              max: 1
            alert:
              title: "WARNING: ArgoCD Uptime alert"
              message: "ArgoCD uptime has dropped below 99%"
              evaluate_every: 1m
              for: 5m
              notify: 1
              tags:
                severity: warning
              on_no_data: no_data
              on_execution_error: alerting
              if:
                - operand: and
                  value: {func: last, ref: A, from: 5m, to: now}
                  threshold: {below: 0.99}

Alerts do not appear under the main alert/list page.
When creating an alert for say a time series panel, the Alert count will go up but no alert is linked to the panel.
Unable to set labels

Time Series

Unable to set the following:

line interpolation
line width
Connect Null Values
Thresholds

Single Stats Panel

Unable to set the following:

Stat styles:
- Orientation
- Text mode
- Text alignment

Misc

Values Map (ex null -> 0 with Color orange)
Single threshold value. Currently requires at least two threshold values. Should this not support the ability to use the "base" threshold and then one extra threshold value?

Dockerfile/Source for kphoen/dark-converter

Hi,
We would like to extend the converter for our use-case.
Anyway we can find the sources anywhere?
Thx.

Vulnerabilities

Trivy scanner shows there are 2 vulnerabilities with possible fix in the latest docker image (kphoen/dark:v0.5.7).

How to reproduce:

Install trivy (https://github.com/aquasecurity/trivy)
Scan: trivy kphoen/dark:v0.5.7

Output:

kphoen/dark:v0.5.7 (alpine 3.14.1)
==================================
Total: 4 (UNKNOWN: 0, LOW: 0, MEDIUM: 2, HIGH: 2, CRITICAL: 0)

+--------------+------------------+----------+-------------------+---------------+--------------------------------------+
|   LIBRARY    | VULNERABILITY ID | SEVERITY | INSTALLED VERSION | FIXED VERSION |                TITLE                 |
+--------------+------------------+----------+-------------------+---------------+--------------------------------------+
| libcrypto1.1 | CVE-2021-3711    | HIGH     | 1.1.1k-r0         | 1.1.1l-r0     | openssl: SM2 Decryption              |
|              |                  |          |                   |               | Buffer Overflow                      |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3711 |
+              +------------------+----------+                   +               +--------------------------------------+
|              | CVE-2021-3712    | MEDIUM   |                   |               | openssl: Read buffer overruns        |
|              |                  |          |                   |               | processing ASN.1 strings             |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3712 |
+--------------+------------------+----------+                   +               +--------------------------------------+
| libssl1.1    | CVE-2021-3711    | HIGH     |                   |               | openssl: SM2 Decryption              |
|              |                  |          |                   |               | Buffer Overflow                      |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3711 |
+              +------------------+----------+                   +               +--------------------------------------+
|              | CVE-2021-3712    | MEDIUM   |                   |               | openssl: Read buffer overruns        |
|              |                  |          |                   |               | processing ASN.1 strings             |
|              |                  |          |                   |               | -->avd.aquasec.com/nvd/cve-2021-3712 |
+--------------+------------------+----------+-------------------+---------------+--------------------------------------+

Update Readme - Questions

It is not clear from the documentation what all datasources are supported. Does it only work with Prometheus? Could someone write a guide as to how to add other datasources or even custom datasources?

Thanks.

Support alerts in converter

Hi there, cool idea and nice work!

I'm evaluating dark for use at various projects and currently running a small PoC. One thing I noticed is that grafana-defined alerts are not converted (using the kphoen/dark-converter:latest image). Same happens for alertlist panel.

Is this on purpose? Mind if I ask should there be any reason for not supporting it, that I may be missing?
Are there any plans on extending the project to extend support to alerts?

I could potentially offer development help and submit a PR if that's something interesting for other use-cases other than mine only :)

Thanks!

Support the definition of notification channels

Grafana 7

Hey,
first nice projekt you got there!

Ich tried dark with the newest grafana version with example dashboard in the k8s directory, but i got the following error:

W0529 12:05:04.550780       1 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0529 12:05:04.554151       1 controller.go:90] Setting up event handlers
I0529 12:05:04.554768       1 controller.go:121] Starting dark-controller
I0529 12:05:04.554787       1 controller.go:124] Waiting for informer caches to sync
I0529 12:05:04.654882       1 controller.go:129] Starting workers
I0529 12:05:04.655155       1 controller.go:135] Started workers
E0529 12:05:04.797732       1 controller.go:243] could not create dashboard from spec: could not find or create folder: invalid character '<' looking for beginning of value
I0529 12:05:04.798103       1 controller.go:200] Successfully synced 'kube-system/example-dashboard'
I0529 12:05:04.799490       1 event.go:278] Event(v1.ObjectReference{Kind:"GrafanaDashboard", Namespace:"kube-system", Name:"example-dashboard", UID:"fe1bdc99-b038-49d7-9bf2-350af06921e8", APIVersion:"k8s.kevingomez.fr/v1", ResourceVersion:"9363847", FieldPath:""}): type: 'Warning' reason: 'Not synced' could not create dashboard from spec: could not find or create folder: invalid character '<' looking for beginning of value

I think this is a downstream problem of a libary you used? Unfortunately I dont know much about go, to fix it myself maybe you can help.

Stat not working for loki datasouce

Stat Not working for loki datasource / targets.
Maybe i missed that part in the documentation, is there somewhere the supported features (grafana ui components) are located?

- stat:
    title: Total
    targets:
      - prometheus:
          query: metric-A{app="app"}

- stat:
    title: Total 2
    datasource: loki
    targets:
      - loki:
          query: metric-B{app="app"}

logs panel type unsupported by dark-converter

2022-08-18T18:04:25Z    WARN    converter/json.go:195   unhandled panel type: skipped   {"type": "logs", "title": "Possible Errors"}
2022-08-18T18:04:25Z    WARN    converter/json.go:195   unhandled panel type: skipped   {"type": "logs", "title": "Full Logs"}
2022-08-18T18:04:25Z    WARN    converter/json.go:195   unhandled panel type: skipped   {"type": "logs", "title": "Debug Logs"}

what else is needed to make this work? i know it was added to grabana, and it ~~might even~~ works on dark, but i'm trying to validate/compare conversion process from existing json dashboard. thanks!

Grafana Service account access

I see the operator explicitly calls out the main way to access your grafana dashboards is through api tokens. Seems like service accounts are also an option for access for the api, is this supported? If not, is this in scope for the future?

Does Dark support alerts on unified alerting?

Hello,

I'm trying to figure out by reading your docs but I'm not getting anywhere.
Is there some type of resource that creates an alert rule in unified alerting?

If not, is there any feature request open for this?

Name confliction

Hello. I am trying to deploy multiple dashboards with alerts using dark. Dark instances are installed across the clusters. The dashboards have same panel, the only difference is datasource. After deploying them, I found that alert rules are reconciling forever. I digged into the problem, and found out that the alert name is determined by the panel title and it cannot be same in the single grafana instance. As a workaround, I am appending some suffix to dashboard/panel titles. I think it would be very helpful if I can override name for dashboard and alerts.

Support graphite as a data source

I'm attempting to migrate to dark. I have several dashboards represented as JSON that include many graphite panels. After some inspection I'm seeing that neither the converter nor grabana has support for graphite as a target.

Despite this arguably being an issue for grabana to add a new graphite target, I'm opening this issue here because I think it may be worth considering solutions alongside grabana for translating yaml dashboard representations to grafana's JSON.

In an ideal world I'd have the ability to specify in my grafanadashboard k8s resource anything I might find in a Grafana dashboard JSON representation and have that work with dark out of the box.

One idea: support an alternative yaml representation that is equivalent to grafana's JSON representation on the grafanadashboard CRD, then transparently send that representation to the grafana API to sync the resource.

I'm very new to dark and grabana, so it's very likely that I'm missing a lot of key information that's leading to my proposal, above. Whatever the right path forward is, I would love to help contribute so that my team can adopt dark!

Just an aside -- I'm really excited about this repo and how it creates the possibility to use git to manage grafana state. Thanks for making it!

skip certificate check

Hello,

I have deployed dark on a k8s cluster and I am self hosting my grafana instance.
Since grafana is exposed with a self signed cert dark is throwing this error:

'Warning' reason: 'Not synced' could not create dashboard from spec: could not find or create folder: Get "https://adc.com/api/folders?limit=100":  x509: certificate signed by unknown authority

Is there anyway to pass a flag to skip checking the certificates?

Thank you

Could not infer datasource UID from its name: stackdriver

Hi ! Thanks for this awesome project, it's excactly what was missing regading dashboards-as-code !

I'm using the Stackdriver datasource, and am using the stackdriver-query.yaml exampledashboard.
When I comment out the alert: key, it works as it should, but with the alert it fails with:

could not create dashboard: could not add new alerts for dashboard: could not infer datasource UID from its name: stackdriver

The operator error msg;

{"level":"error","ts":"2023-03-15T17:22:11Z","msg":"Reconciler error","controller":"grafanadashboard","controllerGroup":"k8s.kevingome
z.fr","controllerKind":"GrafanaDashboard","GrafanaDashboard":{"name":"postgres","namespace":"monitoring"},"namespace":"monitoring","na
me":"postgres","reconcileID":"0020680c-0ec3-42d7-993a-71e4cd0356ec","error":"could not create dashboard: could not add new alerts for 
dashboard: could not infer datasource UID from its name: stackdriver","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/contro
ller.(*Controller).reconcileHandler\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsig
s.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/workspace/vendor/sigs.k8s.io/controller-runt
ime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/
workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"}

What's wrong ?

Btw, the Stackdriver Datasource is defined not with DARK, but with kube-prometheus-stack. But as the datasource works fine for the Dashboard panel, I guess this is not the issue ?

Support the definition of data sources

Support headers/jsonData/secureJsonData for prometheus

Hello!

I would like the ability to pass jsonData and secureJsonData to prometheus, because Mimir is now increasingly used, which needs to pass a header to use multi-tenant metrics.