The purpose of this package is to monitor a number of parameters, for example: number of users created in each second, number of applications created in each second, and so on. The metrics produced can help to detect problem faster. For instance, when the number of users created or applications created goes down drastically. Stakeholders can also refer to these metrics to support future decision.

The project is built on top of prom-client. The client publishes the events, and the subscribers/consumers will consume the events to increment/decrement/set a number of metrics in prometheus client that have been defined before. The next question is how to define the metrics and subscribers/consumers? The client passes file configurations that contain the detailed information. There are two types of file configurations that must be defined.

1. The first one is metric configuration files. In these files, we need to define the prometheus metrics that will be used. Below is the detail of the metric configuration file.

Below is the detail of each metric configuration. You can refer to prom-client for further explanation of each term used here.

Below is the example of a metric configuration file.

{
  "metrics": [
    {
      "name": "app_created_count",
      "help": "app_created_count_help",
      "type": "GAUGE",
      "labels": [
        "productType",
        "partner"
      ]
    }
  ]
}

2. The second one is queue configuration files. In these files, we need to define the consumers/subscribers, for example: what service will be used to implement the consumers/subscribers, the number of consumers/subscribers, what metrics will be affected, and so on. Below is the detail of the queue configuration file.

Below is the detail of each metric configuration.

Below is the example of a queue configuration file that uses GOOGLE_PUB_SUB service. Note that the name inside the metrics field must refer to the metrics that has been defined in step 1. Also, the labelPath contains path to productType and partner which are all of the label names defined in metric app_created_count.

{
  "service": "GOOGLE_PUB_SUB",
  "subscription": "dev_playroom~~event.received.test",
  "consumerCount": 5,
  "concurrency": 4,
  "metrics": [
    {
      "name": "app_created_count",
      "valueModifier": {
        "increase": {
          "path": "numberOfApplicationCreated"
        }
      },
      "labelPath": {
        "productType": "applicationProductType",
        "partner": "applicationPartner"
      }
    }
  ]
}

When the subscriber/consumer receives a new message:

{
   "applicationProductType": "CASH_LOAN",
   "applicationPartner": "INDODANA",
   "numberOfApplicationCreated": 2
}

NOTE: The message format must be in JSON.

The orchestrator will increase metric app_created_count with labels: productType = CASH_LOAN and partner = INDODANA, with value equal to 2.

To use this package you need to instantiate a new orchestrator and pass the required credentials to the constructor depends on the service you use inside the file configurations. Besides, you can also pass the logger object that you want to use to log the error or information to the screen, by default the package uses console. When registering metrics and subscribers/consumers using orchestrator.register, you can pass the metric configuration files to metricFilePaths or metricDirectoryPaths, and pass the queue configuration files to queueFilePaths or queueDirectoryPaths. If you specify the metricDirectoryPaths or queueDirectoryPaths, the program will read the configuration files recursively from each directory. Note that the paths specified in each file path or directory path must be relative to the folder where you run the node server.

require('dotenv').config({ path: process.env.DOTENV_PATH });

const logger = console;

const Orchestrator = require('metric-monitoring');

(async () => {
  const orchestrator = new Orchestrator({
    GOOGLE_PUB_SUB: {
      clientEmail: process.env.CLIENT_EMAIL,
      privateKey: process.env.PRIVATE_KEY,
      projectId: process.env.PROJECT_ID
    },
    AWS: {
      queuePrefixUrl: process.env.QUEUE_PREFIX_URL,
      accessKeyId: process.env.AMAZON_ACCESS_KEY_ID,
      secretAccessKey: process.env.AMAZON_SECRET_ACCESS_KEY,
      region: process.env.AMAZON_SQS_REGION
    }
  }, logger);

  await orchestrator.register({
    queueDirectoryPaths: [
      './resource/queue-config'
    ],
    metricDirectoryPaths: [
      './resource/metric-config'
    ]
  });

  orchestrator.startQueues();

  orchestrator.start(8686, '/metrics');
})();

This code example will serve the resulting metrics at http://localhost:8686/metrics. You can start publishing messages now and see the impact on the metrics.