Gauge metrics suport by Pyrra about pyrra HOT 5 CLOSED

Implemented and closed by @roidelapluie in #598

from pyrra.

@metalmatze, will this be included in the next release? v0.5.6 is missing this support.

from pyrra.

Generally speaking we could and should totally support these time based SLOs as well!

I'm not 100% how the alerting with multi error burn rates would look like.
If we can figure out the PromQL behind these then I'd be totally down for adding this to Pyrra for sure!

from pyrra.

I agree with @rkostyantyn . It would be very useful to support the gauge metrics. I'm not able to track our metrics generated by the blackbox exporter too.

from pyrra.

I've tried implementing an SLO(not using Pyrra) with blackbox-exporter using sum_over_time(probe_success{job="probe"}[5m]) / count_over_time(up{job="probe"}[5m]), but the experience still wasn't great (for my case at least).

My case:

I have multiple VMs that I need to guarantee network connectivity, 1 blackbox-exporter is installed per VM, probing one single enpoint.

sum_over_time(probe_success{job="probe"}[5m]) / count_over_time(up{job="probe"}[5m]) looks good for a single VM, but when rolling out to multiple VM it doesn't work anymore. The reason is because sum_over_time and count_over_time don't aggregate data, so we get multiple timeseries from this query.

Then I tried using sum(probe_success{job="probe"} == 1) / count(up{job="probe"}) and now the query does look good on a graph, but it is tricky to build the recording rules to get Error Budget burns with this one 🤔

from pyrra.