Use managed Prometheus with in-cluster Alertmanager about amazon-managed-service-for-prometheus-roadmap HOT 6 OPEN

a way to use CRDs to configure the rules directly via the Kubernetes APIs

A definite +1 for this!

from amazon-managed-service-for-prometheus-roadmap.

Hey @razvan-moj,

Thank you for posting this issue! I would love to better understand this use case, and had a few follow up questions for you.

1. Have you seen our slack integration via SNS blog post? It walks through how to integrate AMP's alertmanager with Slack such that you have a very similar configuration paradigm to the native slack receiver. The blog can be found here. Would this solve your problem and allow you to use the AMP Alert Manager? If not, we'd love to learn a bit more as to why!

2. We are looking to eventually add support for the native slack receiver. If AMP's alert manager had the native slack receiver support, would that enable you to use the AMP Alert Manager? If not, would love to learn a little bit more about what's driving the use case!

Thank you for submitting this feature proposal! Excited to learn a bit more about the use case from you.

from amazon-managed-service-for-prometheus-roadmap.

Hey @ampabhi-aws !

1. Have you seen our slack integration via SNS blog post? It walks through how to integrate AMP's alertmanager with Slack such that you have a very similar configuration paradigm to the native slack receiver. The blog can be found here. Would this solve your problem and allow you to use the AMP Alert Manager? If not, we'd love to learn a bit more as to why!

We have many (4000 odd as listed above) rules and alerts defined by users in their namespaces; those rules are picked up by Alertmanager once the .yaml is applied. Users do not have access to the AWS API. To change their setup to AMP it seems we need to need to change their workflow from kubectl apply to terraform apply and figure out a way to validate edits so they don't eg overwrite each other, something done now by the namespace isolation and a validation webhook. This work would not be needed if we could just configure the AWS Prometheus to read prometheusrules from the namespaces and hook with the in-cluster Alertmanager.

2. We are looking to eventually add support for the native slack receiver. If AMP's alert manager had the native slack receiver support, would that enable you to use the AMP Alert Manager? If not, would love to learn a little bit more about what's driving the use case!

I don't think this is a problem for us, if we decide to use the AWS API to interact with Prometheus, we can also allow SNS and our users are familiar with how it works.

from amazon-managed-service-for-prometheus-roadmap.

Thanks for the clarification @razvan-moj!

If we provided you a way to use CRDs to configure the rules directly via the Kubernetes APIs, and similarly configure your AMP alert manager via those CRDs, would that alleviate the need for an in-cluster Alertmanager? Trying to get a better sense of if the problem is (1) the AMP Alert manager is inconvenient to use, or (2) Doesn't fit the use case entirely.

from amazon-managed-service-for-prometheus-roadmap.

https://aws.amazon.com/blogs/mt/introducing-the-ack-controller-for-amazon-managed-service-for-prometheus/

from amazon-managed-service-for-prometheus-roadmap.

@ampabhi-aws what are the next steps here?

from amazon-managed-service-for-prometheus-roadmap.