Blocks UI: Find sources for external labels about thanos HOT 8 OPEN

Yes, I meant block stream when I wrote source, likely because we've one block stream per cluster. :)
But besides that we're talking about the same thing, the first column with the block stream index / title label (and now the used space).

I've set up and I am using Thanos in different environments, but came up with the overall architecture, labels, etc... "by myself", haven't seen other Thanos deployments I could compare mine with.

We have Receivers running in each cluster, uploading blocks with 7 labels - 4 of my own (xxx would be our vendor prefix), and 3 coming by default with the Bitnami Thanos Helm chart.

• xxx_cluster: my-cluster-name
• xxx_dc: my-datacenter
• xxx_owner: xxx
• xxx_prod: false
• tenant_id: default-tenant
• receive: true
• receive_replica: xyz-thanos-receive-0

If I'd do it again, I'd be satisfied with adding our custom xxx labels only when remoteWriting from Prometheus to the receiver, but not specifying all of them for the Receiver to label uploaded blocks with.
But even then, at least users of the bitnami Thanos chart are likely to have the default labels tenant_id, receive and receive_replica.
In my case, I'd keep at least a label identifying the cluster as well, so I would've expected most users to have at least 4 external labels - as I said, without knowing other Thanos deployments or the "proper way" of setting it up.

I would be happy to talk about my setup and possible improvements (wasn't that straight forward with OpenShift either...), but I might also be a good example for new Thanos users who might go overboard with their labels.

Copy/pasting the same list presentation form the detail view to the block stream index cell, without changing anything, doesn't work out in my case, even for "higher" streams with multiple resolutions and compaction levels.

I will think about possible designs and prepare some mockups when I find the time for it; ideas I want to play around with:

• smaller font sizes for labels (obviously...)
• using chips / badges and render all labels in a single line instead of a list - like GitHub is doing for example
• maybe adding another, toggleable / collapseable column next to the block stream index for additional block stream info

from thanos.

Here is how it would look without any effort, right after Compactor removed of level of compaction due to retention, resulting in even less height to work with:

Changing the font size alone wouldn't be enough in my case, as I'd need to eliminate some line breaks - that should work out with "GitHub style labels", rendering them in a single row.

But even with a wrapping single row and a smaller font size, hiding some labels could be necessary at some point:

I like your idea of a "Show labels" (or "Show all labels") checkbox to toggle this!

Don't expect anything as quick as the last PR, but I hope I'll find time in April to work on a PR :)

from thanos.

Instead of putting all external labels into the first column / cell, wouldn't it be better to implement a search for labels using the search bar?

@outofrange I believe both are useful. One has a more exploratory nature (render the external labels close to the block stream index) and the other is more investigative (search).

Seeing the labels allows me to acquire more context and potentially identify patterns just by scrolling. Being able to search is useful as I can narrow down my focus, bit I have to already partially know what I am looking for.

from thanos.

The available space next to the source index / label (and my added total size) is quite limited, cramming all external labels in there might get a bit tight, especially if we'd want to show other stats based on the source as well in the future - which I kinda do.

Adding a details pane for the source - like we have for individual blocks, but maybe on the left instead of the right - might give us more space to work with, but then it wouldn't be "Find in page"able anymore.
@douglascamata what do you think?

from thanos.

cramming all external labels in there might get a bit tight

Do you have a deployment of Thanos with many external labels? I've never seen a Thanos deployment with more than 2 external labels myself. Plus there is a lot of space there.

I believe there is a misunderstand: I think we could render the external labels together with the block stream index, on the left side of the block list.

Maybe the picture below will help you understand better. The pink square is my edit. I think we can put the external labels inside it and use more of the real state it has.

from thanos.

Not sure how much you know about Thanos, but by definition all blocks that are in the same block stream (I believe you are calling them "source") share the same external labels name-value pair. Often one of the external labels will mark the source of the data (i.e. thanos ruler 1 or 2, thanos receive 1 or 2, etc) receive and another the tenant. We don't need a lot of space, IMHO.

I like your proposal for the left-side panel, but I don't think we need it yet.

from thanos.

Gotcha. I think we could potentially add a system to hide part of the labels then. Something that shows a few of the labels, potentially sorted alphabetically by label name, and gives you a button to click and see all of them.

I would say to not worry about having a perfect solution. It can be iterated further with time.

from thanos.

Nice screenshots!

I don't like that when the label names + values get too long they will push the beginning of the block stream more to the right and then it doesn't align anymore with the beginning of the other block streams (above or below). Even with the smaller font, at some point this problem will appear again. :/

Maybe even on each line with a label name and its value we will need some mechanism to make longer names look more compacted. We will need a smart solution for both vertical and horizontal space. 🤔

from thanos.