Please add the chart (https://github.com/stakater/IngressMonitorController/tree/master/deployments/kubernetes/chart/ingressmonitorcontroller) to https://github.com/helm/charts/tree/master/stable.

Imho people will look there first for a chart.

Add support for Uptime in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://uptime.com/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Hello there, Ingress builder code is creating wrong monitor url in case there is a named servicePort in ingress.yaml and the path is set as "/*". Basically in the ingress_wrapper.go between L63-L80 is checking for the servicePort in ingress.yaml and adding the servicePort into the monitorURL path, which doesn't look right.

My ingress.yaml looks like this:

kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /healthz
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/security-groups: sg-438f8633
    alb.ingress.kubernetes.io/subnets: subnet-d30c46aa, subnet-b1becafa, subnet-81cce9db
    external-dns.alpha.kubernetes.io/hostname: abc.example.com
    kubernetes.io/ingress.class: alb
    monitor.stakater.com/enabled: "true"
  generation: 2
  labels:
    app: reviewfunnel
    chart: reviewfunnel-0.1.0
    cleanup: "true"
    deployed-with: skaffold
    docker-api-version: "1.39"
    heritage: Tiller
    release: stg-reviewfunnel
    skaffold-builder: local
    skaffold-deployer: helm
    skaffold-tag-policy: sha256
  name: stg-reviewfunnel
  namespace: staging
spec:
  rules:
  - host: abc.example.com
    http:
      paths:
      - backend:
          serviceName: stg-reviewfunnel-app
          servicePort: http
        path: /*

So the monitorURL looks like this
2018/11/21 12:08:38 Monitor URL: http://abc.exmaple.com/http/%2A

Which should be http://abc.example.com/ and when the ingressMonitor service sends a get request, it should return a 200 status code as it's up.

I have raised a PR to fix this issue. The link to the PR is: https://github.com/stakater/IngressMonitorController/pull/132.

Kindly review the PR and let me know about your thoughts

Saw the Medium article, great tool!

We use Pingdom, which also has a decent API (though a fair bit different from the UptimeRobot API).

If this isn't already in the works, I'll have a crack at it 😄 - seems like there might be some changes needed to the httpClient layer to make it happen.

Currently pingdom annotation are in format monitor.stakater.com/pingdom/annotation but this format is not allowed in kubernetes. We have to update the current pingdom annotations to the following format

monitor-name.monitor.stakater.com/annotation

i.e., pingdom.monitor.stakater.com/annotation

ReadMe says file name is imc.yaml whereas in the directory its called ingressmonitorcontroller.yaml

Using the annotation monitor.stakater.com/overridePath with characters like ? currently automatically escapes them: e.g. /health?apikey=test becomes /health%3Fapikey=test and is inserted like this in UptimeRobot monitors.

Typical log output:

Monitor Name: test
Monitor URL: https://test.example.com/health%3Fpwd=test
Monitor Added: test

Is there a way to work around this?

IMC creates several k8s objects while running its tests, and simultaneous tests running causes conflicts as the name of the objects is same.

Append a unique identifier with k8s objects created during tests so that there is no conflict while running multiple pipelines/tests simultaneously.

Add support for Azure Application Insights in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://docs.microsoft.com/en-us/azure/application-insights/app-insights-monitor-web-app-availability
https://azure.microsoft.com/en-us/blog/creating-a-web-test-alert-programmatically-with-application-insights/
https://docs.microsoft.com/en-us/rest/api/application-insights/webtests/createorupdate

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/contributing.md

As suggested by @Anish-Gupta-Bose it will be good to add helm-chart deployments to the Makefile:

helm-template:
	helm template kubernetes/chart/ingressmonitorcontroller --values $(HELMVALUES) --name $(HELMNAME)

helm-install:
	helm install kubernetes/chart/ingressmonitorcontroller --values $(HELMVALUES) --name $(HELMNAME)

Is it possible to run this in a different namespace to the ingess rules it reads?

I see you have to pass the namespace in as an ENV variable https://github.com/stakater/IngressMonitorController/blob/master/src/main.go#L12. But that isn't how most of our ingress controllers are configured.

So it would be nice if this could be deployed in a similar fashion to that, vs every namespace needing a custom ingress monitor controller?

NimSoft is oftenly used for on-prem kubernetes/OpenShift cluster

Add support for pingometer in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://pingometer.com/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Add support for updown in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://updown.io/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Currently we take passwords and api keys as plain configmaps whereas they should be secrets.

Tasks:

• Refactor to convert them to secrets
• Update deployments and examples
• Update readme that its a breaking change and now info should be provided as a secret and not a configmap

Add support for AppBeat in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://www.appbeat.io/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Add support for Stackdriver Uptime Checks and Alert Policies in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

Uptime checks:

• https://cloud.google.com/monitoring/uptime-checks/

Alert Policies:

• https://cloud.google.com/monitoring/uptime-checks/uptime-alerting-policies

Golang client library:

• https://godoc.org/cloud.google.com/go/monitoring/apiv3
• https://godoc.org/google.golang.org/genproto/googleapis/monitoring/v3
• code examples:
  • UptimeCheck example: https://github.com/GoogleCloudPlatform/golang-samples/blob/master/monitoring/uptime/uptime.go
  • Alert Policy example: https://github.com/GoogleCloudPlatform/golang-samples/blob/master/monitoring/alert/alert.go

We must include the udpate strategy by default in all of the manifests e.g.

  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate

Also how many replica's should be run?

Add support for host-tracker in IMC to monitor the uptime of web applications / microservices / REST API's deployed to kubernetes/openshift

https://www.host-tracker.com/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Get IMC listed in the public helm repository i.e. https://github.com/helm/charts

Regarding documentation of one IMC for all namespaces:

• clearly specify in the readme
• add an example in k8s manifest folder

test issue

Hi there, I have a follow use case - have Ingress controller installed and below is its sample resource file:

annotations:
   kubernetes.io/ingress.class: "nginx"
   monitor.stakater.com/enabled: "true"
Spec:
rules:
  - host: helloworld.awesome-customer.com
    http:
    paths:
    - path: /*
    backend:
    serviceName: helloworld
    servicePort: 8080
  - host: nextfineapp.awesome-customer.com
    http:
    paths:
    - path: /*
    backend:
    serviceName: nextfineapp
    servicePort: 8080

I would like to configure IMC with Pingdom as service provider. In this case IMC will monitor this particular Ingress controller. However, what I need is to receive notification in case particular service is down. So I need per service notification. How can I achieve that?

Would you accept a PR with a ClusterRole instead of the current Role, or an option to chose between the two in the helm release?

Our use case is that we have a tooling namespace for all controllers and do not duplicate those controllers for each namespace.

Suggestion from @Mikulas

Refactor the code; namely the operator work queue would be easier to work with if it was in fact a map with the *Ingress object. It would also simplify the code as it would remove the parsing (splitting) of the namespace/name key and would eliminate the API calls for getting an Ingress from key (name).

Currently, hard code values for the status code are being used like 200 and 404 etc, which is not easy to understand(for beginners) and error-prone(because we might need to updated all files for a single status code change).

OLD SOLUTION

Instead of hard code values, constants must be used. A map has been defined in the pkg/constants that contains the constants for status codes.

filename: httpStatusCodesMap.go

var StatusCodes = map[string]int{
	"OK":            200,
	"NOT_FOUND":     404,
	"UN_AUTHORISED": 401,
	"BAD_REQUEST":   400,
	"CREATED":       201}

NEW SOLUTION

Based on @waseem-h suggestion, go's net/http package constants will be used for response codes.

This change must be applied to all IMC modules.

uptimerobot.com supports adding basic auth (http_username & http_password) to monitors. It would be nice, if IngressMonitorController could set those settings via related Ingress annotations/secrets.

Howdy! Thanks for putting this controller together, it exactly solves the problem I'm working on at $WORK, so I'm quite grateful for all your hard work. I've noticed a few issues in our use case with the way the monitor URL is constructed, and I'd be happy to open PRs for either of these if you're interested.

• protocol: our ingress doesn't handle TLS on its own (we're running behind an ALB which terminates SSL, and using weave in our cluster for transparent pod-to-pod encryption), so our ingress manifest simply declares http endpoints. as a result, the logic for determing the protocol thinks we're doing plain http and uses the result from iw.getHost(), even though the actual endpoint is HTTPS. https://github.com/stakater/IngressMonitorController/blob/master/src/ingress-wrapper.go#L81-L85

• ingressPort: in the ingress, we define our service port by name, and reference it accordingly in the manifests for the ingress and service:

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: test-app
    spec:
      rules:
      - host: example.host
        http:
          paths:
          - backend:
              serviceName: test-app
              servicePort: http
            path: /
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: test-app
    spec:
      ports:
      - name: http
        port: 80
        protocol: TCP
        targetPort: http
      selector:
        app: test-app
      type: ClusterIP
  

  As a result, the monitor url looks like http://example.host/http/healthcheck, when it could either be http://example.host:80/healthcheck, or simply http://example.host/healthcheck. https://github.com/stakater/IngressMonitorController/blob/master/src/ingress-wrapper.go#L95-L96

• healthcheck path: I noticed there's a monitorHealthAnnotation annotation that presumably I could put on my ingress to specify the healthcheck path, but it looks like it only gets appended to the existing u.Path value. One way to solve the ingressPort issue for us would be if this annotation /replaced/ the entire u.Path value, and it would also give more control over the healthcheck in general, if interested... https://github.com/stakater/IngressMonitorController/blob/master/src/ingress-wrapper.go#L107-L110

If you're interested in accepting PRs for customizing these, I imagine they could be top level options in the config map, or per-resource annotation overrides, depending on how you'd prefer to handle the options. And, if you're not interested in accepting incidental complexity, then no problem - we'll fork and go on our merry way. Thanks again!

Kubernetes Version: 1.10.8
Ingress Monitor Controller Version: 1.0.26
Provider: Uptime Robot

Deleted a service and ingress with IMC annotation, but it did not delete the monitor in Uptime Robot.
The logs for IMC were as follows:

2018/11/05 12:38:24 Get service from kubernetes cluster error:services "example-ws" not found
2018/11/05 12:38:24 Monitor Name: prod-example-ws
2018/11/05 12:38:24 Monitor URL: https://example.com/public/example/0.1
2018/11/05 12:38:24 Monitor already exists for ingress: prod-example-ws
2018/11/05 12:38:24 Monitor Updated: prod-example-ws

imc- is not appended to ServiceAccountName template in helpers.tpl so it will cause issue when deploying through Umbrella charts

Currently, I notice there is an out directory which contains the binary data. Could we make use of GitHub releases to manage this rather?

Couple of things are misleading/confusing in the README:

• it must clearly specify that IMC can be run per namespace or globally
• user doesn't need to download the repo; rather just run with -f and provide file path's
• separate the installation with vanilla manifests and helm charts in more cleaner way

Nevermind.

We must document what naming convention to follow for naming common and specific annotations.

After creating the monitors, if one makes changes to the Ingress annotations related to the IngressMonitorController, they aren't reflected back to the Monitoring service.

Add support for Uptrends in IMC to monitor the uptime of web applications / microservices deployed to kubernetes/openshift

https://www.uptrends.com/products/website-monitoring

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

Currently, it seems that IngressMonitorController discovers the URLs to be monitored only at startup or on Ingress changes.

If the readiness URLs of the pods behind a ingress changes, the URLs are not updated automatically.

The issue could be solved by doing regularly a full-scan of all monitors and update the URLs if necessary.
My suggestion would be to try simply a resyncPeriod > 0 in

• Make the README very lean, and move everything else to documentation
• Fix banner

To simplify the installation of IMC there should be one k8s manifests which includes all k8s resources/objects i.e. rbac, configmap, deployment and name it imc.

And then add in README how the user can just use that one manifest to install IMC

I'm trying out IngressMonitorController with StatusCake for my K8s cluster.

It appears that oftentimes an ingress entity is created, it is processed twice by the monitor, resulting in two identical tests being added to StatusCake.

I'm using Kubernetes Server v1.11.3

There does appear to be an error in the log when this occurs:

2018/10/09 16:11:55 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:11:56 Monitor Added: 3534589
2018/10/09 16:11:56 Monitor Name: jnf-app1
2018/10/09 16:11:56 Monitor URL: https://test-jnf-app1.testdomain.com/app_status
2018/10/09 16:11:56 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:11:57 Monitor Added: 3534590
2018/10/09 16:11:57 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:11:57 Cannot find monitor with name: jnf-app2
2018/10/09 16:11:57 Monitor Name: jnf-app2
2018/10/09 16:11:57 Monitor URL: https://test-jnf-app2.testdomain.com/app_status
2018/10/09 16:11:57 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:11:58 Monitor Added: 3534591
2018/10/09 16:11:58 Monitor Name: jnf-app2
2018/10/09 16:11:58 Monitor URL: https://test-jnf-app2.testdomain.com/app_status
2018/10/09 16:11:59 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:11:59 Monitor Added: 3534592
2018/10/09 16:13:00 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:13:00 Cannot find monitor with name: testdomain-projects
2018/10/09 16:13:24 Monitor Name: projects
2018/10/09 16:13:24 Monitor URL: https://test-projects.testdomain.com/app_status
2018/10/09 16:13:25 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:13:26 Monitor Added: 3534595
2018/10/09 16:13:28 Monitor Name: projects
2018/10/09 16:13:28 Monitor URL: https://test-projects.testdomain.com/app_status
2018/10/09 16:13:28 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:13:29 Monitor Added: 3534596
2018/10/09 16:15:11 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:15:11 Cannot find monitor with name: jnf-app2
2018/10/09 16:15:44 Monitor Name: jnf-app2
2018/10/09 16:15:44 Monitor URL: https://test-jnf-app2.testdomain.com/app_status
2018/10/09 16:15:45 json: cannot unmarshal number 42.86 into Go struct field StatusCakeMonitorMonitor.Uptime of type int
2018/10/09 16:15:45 Monitor Added: 3534601
2018/10/09 16:15:57 Monitor Name: jnf-app2
2018/10/09 16:15:57 Monitor URL: https://test-jnf-app2.testdomain.com/app_status
2018/10/09 16:15:58 Monitor already exists for ingress: jnf-app2
2018/10/09 16:15:59 Monitor couldn't be updated: jnf-app2
2018/10/09 16:15:59 No data has been updated (is any data different?)

StatusCake is another widely used downtime checker/notifier and we should include its support in IMC.

Add support for Site24x7 in IMC to monitor the uptime of web applications / microservices / REST API's deployed to kubernetes/openshift

https://www.site24x7.com/

Conventions to follow: https://github.com/stakater/IngressMonitorController/blob/master/docs/developing/conventions.md

PR's are very welcome

The check api definitions are here:

https://www.pingdom.com/api/2.1/#MethodCreate+New+Check

Basically, we need to be able to specify the integration ids for checks to allow checks created by the controller to have integration features.

integrationids | Integration identifiers. For example integrationids=11,22,33 | Comma separated Integers

The work is minimal, so I will see if I can submit a PR unless someone more familiar wants to do it.

Hi,

Good work for IngressMonitor,

I'm a StatusCake user, so I created an API key, and deployed configmap, rbac and Deployment of IngressMonitorController v1.0.6,

kind: ConfigMap
metadata:
  annotations:
    fabric8.io/target-platform: kubernetes
  labels:
    app: ingressmonitorcontroller
    version: 1.0.6
    group: com.stakater.platform
    provider: stakater
    chart: "IngressMonitorController-1.0.6"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: ingressmonitorcontroller
data:
  config.yaml: |-
    providers:
    - name: UptimeRobot
      apiKey: <myKEY>
      apiURL: https://app.statuscake.com/API/
      alertContacts: Principal
    enableMonitorDeletion: true
    monitorNameTemplate: "{{.Namespace}}-{{.IngressName}}"

Small problem,

It start and stuck at Starting Ingress Monitor controller ...

So I think, I need to create an annotation for my ingresses now, and it doesn't change anything.

Thanks for debuging this with me.

(I work in IPv6 environment.)

We must add a section in README which lists downs the k8s version on IMC has been tested and is being used

It would be great to support https://victorops.com/ for liveness alert creation.

For installing IMC using Helm we still mention that user must clone and copy etc. but now we do have public helm charts git repo https://github.com/stakater/stakater-charts

So, we must specify how easily can user do the installation using that helm charts repo.

The IngressMonitorController seems to append an extra slash into the URL.

If the health endpoint of an pod is set to /myapp/health and the path of a corresponding ingress is set to '/', the final requested URL contains two slashes: https://example.com//myapp/health.

This breaks at least modern spring applications (they return http 500 with "The request was rejected because the URL was not normalized.").

A workaround is to specify the ingress without path: /.

Currently we support Ingress only and we should add support for Routes as well.

As suggested by The annotation monitor.stakater.com/pingdom/resolution can be abstracted into a general annotation such as monitor.stakater.com/resolutionTime as it can be used with premium services of uptimerobot and probably other services providers too.

Make it the annotation common and document that it is currently supported with pingdom only, we can later add support for resolution time for uptimerobot as well (Currently uptimerobot does not allow specifying resolution times manually for free services)