grafana / intro-to-mltp Goto Github PK

Hi I have Grafana on-prem, is it possible or how I can run your docker images except Grafana and point all images to my grafana?
Thanks

Hello. I am wondering why there's a multiplication by 55 in MLT dashboard in Request Rate panel, here:

          "expr": "(sum by (http_target)(rate(traces_spanmetrics_latency_sum{http_target=~\"${httpEndpoint}\",service_version=~\"${serviceVersion}\"}[1m]))) / (sum by (http_target)(rate(traces_spanmetrics_latency_count{http_target=~\"${httpEndpoint}\",service_version=~\"${serviceVersion}\"}[1m]))) * 55",

Thank you for your reply!

Not sure if this is intended but the database pod logs show a steady stream of 'violates not-null constraint' errors

│ 2024-02-29 15:20:13.081 UTC [67] ERROR: null value in column "name" of relation "illithid" violates not-null constraint ││ 2024-02-29 15:20:13.081 UTC [67] DETAIL: Failing row contains (726, null). ││ 2024-02-29 15:20:13.081 UTC [67] STATEMENT: INSERT INTO illithid(name) VALUES ($1) ││ 2024-02-29 15:20:16.259 UTC [70] ERROR: null value in column "name" of relation "owlbear" violates not-null constraint ││ 2024-02-29 15:20:16.259 UTC [70] DETAIL: Failing row contains (753, null). ││ 2024-02-29 15:20:16.259 UTC [70] STATEMENT: INSERT INTO owlbear(name) VALUES ($1) ││ 2024-02-29 15:20:17.631 UTC [110] LOG: invalid length of startup packet ││ 2024-02-29 15:20:20.862 UTC [67] ERROR: null value in column "name" of relation "unicorn" violates not-null constraint ││ 2024-02-29 15:20:20.862 UTC [67] DETAIL: Failing row contains (763, null). ││ 2024-02-29 15:20:20.862 UTC [67] STATEMENT: INSERT INTO unicorn(name) VALUES ($1) ││ 2024-02-29 15:20:24.071 UTC [69] ERROR: null value in column "name" of relation "beholder" violates not-null constraint ││ 2024-02-29 15:20:24.071 UTC [69] DETAIL: Failing row contains (795, null). ││ 2024-02-29 15:20:24.071 UTC [69] STATEMENT: INSERT INTO beholder(name) VALUES ($1) ││ 2024-02-29 15:20:27.447 UTC [67] ERROR: null value in column "name" of relation "beholder" violates not-null constraint ││ 2024-02-29 15:20:27.447 UTC [67] DETAIL: Failing row contains (797, null). ││ 2024-02-29 15:20:27.447 UTC [67] STATEMENT: INSERT INTO beholder(name) VALUES ($1) ││ 2024-02-29 15:20:28.172 UTC [67] ERROR: null value in column "name" of relation "beholder" violates not-null constraint ││ 2024-02-29 15:20:28.172 UTC [67] DETAIL: Failing row contains (798, null). ││ 2024-02-29 15:20:28.172 UTC [67] STATEMENT: INSERT INTO beholder(name) VALUES ($1) ││ 2024-02-29 15:20:30.419 UTC [70] ERROR: null value in column "name" of relation "unicorn" violates not-null constraint ││ 2024-02-29 15:20:30.419 UTC [70] DETAIL: Failing row contains (768, null). ││ 2024-02-29 15:20:30.419 UTC [70] STATEMENT: INSERT INTO unicorn(name) VALUES ($1)

Needs to update this example to use Pyroscope here (and also rename all mentions of phlare to Pyroscope)

If I checkout this repo and run docker compose up I get this error:

Pulling mythical-server (grafana/intro-to-mltp:mythical-beasts-server-latest)...
ERROR: manifest for grafana/intro-to-mltp:mythical-beasts-server-latest not found: manifest unknown: manifest unknown

If I edit the docker-compose.yml file like so:

  mythical-requester:
-    image: grafana/intro-to-mltp:mythical-beasts-requester-latest
+    image: grafana/intro-to-mltp:mythical-beasts-requester-0.3.3

  mythical-server:
-    image: grafana/intro-to-mltp:mythical-beasts-server-latest
+    image: grafana/intro-to-mltp:mythical-beasts-server-0.3.3

  mythical-recorder:
-    image: grafana/intro-to-mltp:mythical-beasts-recorder-latest
+    image: grafana/intro-to-mltp:mythical-beasts-recorder-0.3.3

It works as expected.

Is the docker hub tag broken/missing from https://hub.docker.com/r/grafana/intro-to-mltp/tags ?

Switch to native OTLP ingest and apply bloom filters.

At the moment exemplars are always pinned to the floor because of the short latencies, add customer histogram values.

The example perfectly shows the capabilities of the stack in action. However, there are problems in reproducing the same results in kubernetes cluster. It would be great if there were pre-configured Helm charts or yaml manifests To reproduce all the same features in Kubernetes.

I will prefix this by saying I am not sure if this is an actual issue or by design.

However, in the "95th Percentile Response Latencies (ms)" Panel, recently it shows a value of 1000 for all Endpoints, which while it appears to be correct, does not feel expected or realistic.

I suspect this is to do with a change added here;

#75

To test this, I used the latest Image Tag on Docker (which I believe relates to 0.3) and then against 0.2;

From around ~10:30 to 11:00am on that Graph, its using the latest image tag and has a consistent value of 100 for all endpoints.

At ~11:00am, I switched the Image Tag to 0.2 and restarted everything, and from then the values drop right down, and appear to vary quite a bit;

I suspect the spiky graph feature that was added is what is causing the latency to always show a value of 1000, this is fine, but it does not feel as realistic as before.

Am wondering if there is a change that could be made here to make this look a bit more "real" or similar.

If its expected and not an issue, then feel free to close this.

Thanks!

Your Docker Compose uses PWD for volumes. When I run it on my mac, docker compose up, an appropriate warning comes up:

docker compose up
WARN[0000] The "PWD" variable is not set. Defaulting to a blank string.
WARN[0000] The "PWD" variable is not set. Defaulting to a blank string.
WARN[0000] The "PWD" variable is not set. Defaulting to a blank string.
WARN[0000] The "PWD" variable is not set. Defaulting to a blank string.
[+] Running 7/0
 ⠿ Container intro-to-mlt-mythical-database-1   Running                                                                        0.0s
 ⠿ Container intro-to-mlt-loki-1                Running                                                                        0.0s
 ⠿ Container intro-to-mlt-mythical-server-1     Running                                                                        0.0s
 ⠿ Container intro-to-mlt-grafana-1             Running                                                                        0.0s
 ⠿ Container intro-to-mlt-mythical-queue-1      Running                                                                        0.0s
 ⠿ Container intro-to-mlt-mythical-requester-1  Running                                                                        0.0s
 ⠿ Container intro-to-mlt-mythical-recorder-1   Running                                                                        0.0s
Attaching to intro-to-mlt-agent-1, intro-to-mlt-grafana-1, intro-to-mlt-loki-1, intro-to-mlt-mimir-1, intro-to-mlt-mythical-database-1, intro-to-mlt-mythical-queue-1, intro-to-mlt-mythical-recorder-1, intro-to-mlt-mythical-requester-1, intro-to-mlt-mythical-server-1, intro-to-mlt-phlare-1, intro-to-mlt-tempo-1
intro-to-mlt-mythical-recorder-1   | Received a message: POST /illithid {"name":"blush_Willi_dingo"}
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884073339Z caller=table_manager.go:134 msg="uploading tables"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884111422Z caller=index_set.go:86 msg="uploading table index_19499"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884116297Z caller=index_set.go:107 msg="finished uploading table index_19499"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884120006Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19499"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884123006Z caller=index_set.go:86 msg="uploading table index_19496"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884126089Z caller=index_set.go:107 msg="finished uploading table index_19496"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884128297Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19496"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884133839Z caller=index_set.go:86 msg="uploading table index_19497"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884135672Z caller=index_set.go:107 msg="finished uploading table index_19497"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884137881Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19497"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884139881Z caller=index_set.go:86 msg="uploading table index_19498"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884141631Z caller=index_set.go:107 msg="finished uploading table index_19498"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.884145297Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19498"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910569797Z caller=table_manager.go:166 msg="handing over indexes to shipper"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910630172Z caller=table.go:318 msg="handing over indexes to shipper index_19496"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910637589Z caller=table.go:334 msg="finished handing over table index_19496"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910660506Z caller=table.go:318 msg="handing over indexes to shipper index_19497"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910664797Z caller=table.go:334 msg="finished handing over table index_19497"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910675839Z caller=table.go:318 msg="handing over indexes to shipper index_19498"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910678006Z caller=table.go:334 msg="finished handing over table index_19498"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910690589Z caller=table.go:318 msg="handing over indexes to shipper index_19499"
intro-to-mlt-loki-1                | level=info ts=2023-05-22T13:39:33.910692714Z caller=table.go:334 msg="finished handing over table index_19499"
intro-to-mlt-mythical-recorder-1   | Received a message: POST /beholder {"name":"tan_Neysa_pinniped"}
intro-to-mlt-mythical-recorder-1   | Received a message: GET /manticore
intro-to-mlt-mythical-recorder-1   | Received a message: POST /beholder {"name":"copper_Kameko_caribou"}
intro-to-mlt-mythical-database-1   | 2023-05-22 13:39:34.454 UTC [35] ERROR:  null value in column "name" of relation "illithid" violates not-null constraint
intro-to-mlt-mythical-database-1   | 2023-05-22 13:39:34.454 UTC [35] DETAIL:  Failing row contains (6454, null).
intro-to-mlt-mythical-database-1   | 2023-05-22 13:39:34.454 UTC [35] STATEMENT:  INSERT INTO illithid(name) VALUES ($1)
intro-to-mlt-mythical-recorder-1   | Received a message: DELETE /beholder pink_Alyssa_tarantula
Error response from daemon: error while creating mount source path '/host_mnt/tempo/tempo.yaml': mkdir /host_mnt/tempo: input/output error

Instead, I needed to run it as: PWD=$PWD docker compose up. Not a big deal and is Docker related, but I wonder if it can be mentioned in the README for others to know.

Running it on M1 Mac.

It's high-time. There's nothing here that's really going to cause issues, we are not an upstream dependency for anyone.
But we should refresh versions and ensure that we're mitigating Dependabot complaints. :)

I checked out the repo and trying to run the demo env using docker compose up on M1 Mac OS (13.4.1). The docker build never completes.

Docker version 20.10.18
npm version 9.6.4

=> [intro-to-mlt-mythical-requester internal] load build definition from Dockerfile                                                                                              0.0s
 => => transferring dockerfile: 943B                                                                                                                                              0.0s
 => [intro-to-mlt-mythical-recorder internal] load build definition from Dockerfile                                                                                               0.0s
 => => transferring dockerfile: 943B                                                                                                                                              0.0s
 => [intro-to-mlt-mythical-server internal] load build definition from Dockerfile                                                                                                 0.0s
 => => transferring dockerfile: 943B                                                                                                                                              0.0s
 => [intro-to-mlt-mythical-requester internal] load .dockerignore                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                   0.0s
 => [intro-to-mlt-mythical-recorder internal] load .dockerignore                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                                   0.0s
 => [intro-to-mlt-mythical-server internal] load .dockerignore                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                   0.0s
 => [intro-to-mlt-mythical-recorder internal] load metadata for docker.io/library/node:17-alpine3.14                                                                              0.0s
 => [intro-to-mlt-mythical-requester internal] load build context                                                                                                                 0.0s
 => => transferring context: 10.48kB                                                                                                                                              0.0s
 => [intro-to-mlt-mythical-recorder builder 1/6] FROM docker.io/library/node:17-alpine3.14                                                                                        0.0s
 => CACHED [intro-to-mlt-mythical-recorder builder 2/6] WORKDIR /usr/src/app                                                                                                      0.0s
 => CACHED [intro-to-mlt-mythical-requester builder 3/6] RUN apk add --update --no-cache python3 alpine-sdk && ln -sf python3 /usr/bin/python &&     python3 -m ensurepip &&      0.0s
 => CACHED [intro-to-mlt-mythical-requester builder 4/6] COPY mythical-beasts-requester/package.json /usr/src/app/                                                                0.0s
 => CACHED [intro-to-mlt-mythical-requester builder 5/6] RUN apk update && apk upgrade &&     apk add --no-cache bash git openssh                                                 0.0s
 => [intro-to-mlt-mythical-requester builder 6/6] RUN npm install --production     && npm cache clean --force     && rm -rf /tmp/*                                              353.3s
 => => # npm WARN config production Use `--omit=dev` instead.
 => [intro-to-mlt-mythical-server internal] load build context                                                                                                                    0.0s
 => => transferring context: 19.18kB                                                                                                                                              0.0s
 => CACHED [intro-to-mlt-mythical-server builder 3/6] RUN apk add --update --no-cache python3 alpine-sdk && ln -sf python3 /usr/bin/python &&     python3 -m ensurepip &&     pi  0.0s
 => CACHED [intro-to-mlt-mythical-server builder 4/6] COPY mythical-beasts-server/package.json /usr/src/app/                                                                      0.0s
 => CACHED [intro-to-mlt-mythical-server builder 5/6] RUN apk update && apk upgrade &&     apk add --no-cache bash git openssh                                                    0.0s
 => [intro-to-mlt-mythical-server builder 6/6] RUN npm install --production     && npm cache clean --force     && rm -rf /tmp/*                                                 353.3s
 => => # npm WARN config production Use `--omit=dev` instead.
 => [intro-to-mlt-mythical-recorder internal] load build context                                                                                                                  0.0s
 => => transferring context: 330B                                                                                                                                                 0.0s
 => CACHED [intro-to-mlt-mythical-recorder builder 3/6] RUN apk add --update --no-cache python3 alpine-sdk && ln -sf python3 /usr/bin/python &&     python3 -m ensurepip &&       0.0s
 => CACHED [intro-to-mlt-mythical-recorder builder 4/6] COPY mythical-beasts-recorder/package.json /usr/src/app/                                                                  0.0s
 => CACHED [intro-to-mlt-mythical-recorder builder 5/6] RUN apk update && apk upgrade &&     apk add --no-cache bash git openssh                                                  0.0s
 => [intro-to-mlt-mythical-recorder builder 6/6] RUN npm install --production     && npm cache clean --force     && rm -rf /tmp/*                                               353.3s
 => => # npm WARN config production Use `--omit=dev` instead.