Code Monkey home page Code Monkey logo

jmeter-dynatrace-plugin's Introduction

Dynatrace Backend listener plugin for JMeter

This library implements a JMeter Backend listener and sends the recorded loadtest metrics via the Dynatrace MINT metric ingest to the configured Dynatrace monitoring environment.

See https://jmeter.apache.org/usermanual/component_reference.html#Backend_Listener for a description of the JMeter BackendListener concept.

Building

Execute the gradle build task. This will generate a "jmeter-dynatrace-plugin-VERSION" in the build/libs directory

Prerequisites

JMeter 5.3 (https://jmeter.apache.org/download_jmeter.cgi)

Dynatrace Version > 1.202

Installation

Copy this generated jar to the apache-jmeter-5.3\lib\ext folder and (re-)start JMeter

Maven repository

The plugin is now also available in maven central

See https://search.maven.org/artifact/com.dynatrace.jmeter.plugins/jmeter-dynatrace-plugin/1.8.0/jar

Configuration

Dynatrace

  • Open the WebUI of your monitoring environment
  • Go to Settings > Integration > Dynatrace API
  • Select Generate token
  • Add a token name and select the access scope Ingest metrics using API V2
    • If you also want to read those metrics, you can also select Read metrics using API V2
  • Copy the generated token ad store it in a secure space (it will be needed for the JMeter plugin configuration)

JMeter

  • Open your JMeter jmx file
  • Add Listener > Backend Listener
  • Select Backend Listener implementation: com.dynatrace.jmeter.plugins.MintBackendListener
  • Change the required parameters:
    • dynatraceMetricIngestUrl: the URL of your monitoring environment with the suffix /api/v2/metrics/ingest
    • dynatraceApiToken: the API token which you generated for your Dynatrace API integration
    • transactionDimensions: a comma-separated list of key=value pairs which will be used as dimensions for the test step related metrics. You must either replace the "dt.entity.service=SERVICE-XXXXXXXXXXXXX" with the correct service id or remove/replace the parameter.
  • Change the optional parameters if needed:
    • testDimensions: a comma-separated list of key=value pairs which will be used as dimensions for the test related metrics.
    • enabled: allows to disable this plugin from outside without changing the jmeter script itself by passing the parameter -Jenabled=false to the start script.
    • name: The name of the listener which will be used as prefix for the log messages. If you have multiple listeners for multiple thread groups you should use unique names.
    • samplersRegex: a regular expression which allows to filter the samplers which generate the metrics
  • Start the load test

Debug logging

If you want to enable debug logs for this plugin, please add the line <Logger name="com.dynatrace.jmeter.plugins" level="debug" /> to the log4j2.xml file in the JMeter bin folder.

MINT metrics

When the JMeter test is running, it will generate the specified general metrics:

  • jmeter.usermetrics.startedthreads: the number of started threads
  • jmeter.usermetrics.finishedthreads: the number of finished threads
  • jmeter.usermetrics.minactivethreads: the minimum number of active threads
  • jmeter.usermetrics.maxactivethreads: the maximum number of active threads
  • jmeter.usermetrics.meanactivethreads: the arithmetic mean of active threads

Dimensions used for those metrics:

  • testDimensions: a comma-separated list of key=value pairs which will be used as dimensions for the test related metrics. e.g. dimension1=Test1,dimension2=Test2

In addition it will generate the specified metrics for each test step (JMeter sampler)

  • jmeter.usermetrics.transaction.mintime: the minimal elapsed time for requests within sliding window
  • jmeter.usermetrics.transaction.maxtime: the maximal elapsed time for requests within sliding window
  • jmeter.usermetrics.transaction.meantime: the arithmetic mean of the elapsed time
  • jmeter.usermetrics.transaction.receivedbytes: the number of received bytes
  • jmeter.usermetrics.transaction.sentbytes: the number of sent bytes
  • jmeter.usermetrics.transaction.hits: the number of hits to the server
  • jmeter.usermetrics.transaction.error: the number of failed requests
  • jmeter.usermetrics.transaction.success: the number of successful requests
  • jmeter.usermetrics.transaction.count: the total number of requests

Dimensions used for those metrics:

  • transactionDimensions: a comma-separated list of key=value pairs which will be used as dimensions for the test step related metrics. e.g. dimension3=Test3,dimension4=Test4

Monitoring the metrics in Dynatrace

  • Create a dashboard
  • For every metric:
    • Add a custom chart
    • Select Try it out in the Banner line Analyze multidimensional metrics from Prometheus, StatsD and others channels right on your dashboards
    • Enter the name of the metric (from the list of the metrics above) in the field Filter metrics by...
    • Specify the chart settings (Visualization type, Chart type...)
    • Select Pin to dashboard
  • Save the dashboard
    `

jmeter-dynatrace-plugin's People

Contributors

christian-inzko-dt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

jmeter-dynatrace-plugin's Issues

Delete Parameters in Backend Listener not working

Deleting any parameter in the backend listener, not actually deleting it.

Steps to reproduce:

  1. Add a backend listener and select com.dynatrace.jmeter.plugins.MintBackendListener
  2. Delete any parameter(s)
  3. Click on any other element in the test plan
  4. Click on the backend listener again, you could see the deleted parameters are added again.

Dynatrace Metrics not matching with JMeter

I did a simple load test against my local app. I have found conflicts between the dynatrace metrics and JMeter metrics.

JMeter Summary Report
QAInsights-0078

Conflicting Metrics:

  Transaction Name JMeter Dynatrace
jmeter.usermetrics.transaction.maxtime T00_HomePage 1730 141
  T10_TopCommand 8029 429
jmeter.usermetrics.transaction.mintime T00_HomePage 5 5
  T10_TopCommand 7 8
jmeter.usermetrics.transaction.meantime: T00_HomePage 24 13.6
  T10_TopCommand 94 28.6
jmeter.usermetrics.transaction.receivedbytes T00_HomePage 355.14 KB/sec 109mil
  T10_TopCommand 356.71 KB/sec 109mil
jmeter.usermetrics.transaction.sentbytes T00_HomePage 1.86/s 569k
  T10_TopCommand 2.087/s 637k

400 error on plugin startup for first POST of metrics to the endpoint.

I've implemented the plugin on my companies jmeter tests and everything seems to report properly during the test. 1 thing that I can't figure out is we consistently get a 400 error when the listener is starting up and configuring itself for the first time.

In the Jmeter logs we get a WARN(ing) on the first time it submits:
image

Does this matter? Seems like a startup order issue. There are not metrics to send at that time.

I've tried altering my payload and making small changes but no luck. Otherwise it works well and sends in the metrics for the rest of the test.

Plugin dependencies must have "provided" not "runtime" scope

When we add the plugin as a maven dependency, it breaks our code (which needs JMeter dependencies like ApacheJmeter_Core:

    <dependency>
      <groupId>org.apache.jmeter</groupId>
      <artifactId>ApacheJMeter_http</artifactId>
      <scope>provided</scope>
    </dependency>
<!-- Breaks the module compilation when added -->
    <dependency>
      <groupId>com.datadoghq</groupId>
      <artifactId>jmeter-datadog-backend-listener</artifactId>
      <scope>runtime</scope>
    </dependency>

Datadog works fine as the plugin dependencies are correctly scoped as provided:
https://github.com/DataDog/jmeter-datadog-backend-listener

As of now, we're forced to add ApacheJmeter_Core as provided dependency to our pom.xml as a workaround to fix this issue:

    <dependency>
      <groupId>com.dynatrace.jmeter.plugins</groupId>
      <artifactId>jmeter-dynatrace-plugin</artifactId>
      <scope>runtime</scope>
    </dependency>
    <!-- Required due to dynatrace plugin "runtime" dependencies -->
    <dependency>
      <groupId>org.apache.jmeter</groupId>
      <artifactId>ApacheJMeter_core</artifactId>
      <version>${jmeter.version}</version>
      <scope>provided</scope>
    </dependency>

Please change the scope to provided instead of runtime to avoid breaking the compilation classpath of modules which need your plugin.

Introduce a parameter to enable/disable the plugin

I would like to introduce a new parameter called "enable". The default can be true - but - I would like to use this parameter to enable/disable this extension via a JMeter parameter instead of having to enable/disable it through a JMX setting.

My use case is that we leverage JMeter with the open source project keptn (www.keptn.sh). When uploading a JMX File that contains that Dynatrace plugin I would like to enable/disable this extension via a parameter that Keptn passes to JMeter. This gives us more flexibility to control the behavior instead of having to upload different versions of the same JMX file

MintMetricSender.checkConnection has short timeout and does not log error nicely

I was getting many errors like this:
2021-01-16 00:07:24,990 INFO c.d.j.p.MintBackendListener: Start MINT metric sender for url https://zzzz.live.dynatrace.com/api/v2/metrics/ingest failed with General Error executing connection check for MINT server, setting enabled state to false

I tracked it down to a lastRequest.get() call in MintMetricSender.checkConnection() that has a timeout of only 1 second. By changing the timeout to 30 seconds, I am able to avoid the errors.

Also, unless I set loglevel to DEBUG, it doesn't tell me that it is a timeout.

The exception-catching code looks like this

		catch (ExecutionException | TimeoutException | InterruptedException ex) {
			log.debug("Error executing connection check for MINT server", ex);
			throw new MintConnectionException("General Error executing connection check for MINT server");
		}

I would suggest you log.warn, or log.error, the exception type, And of course, make the timeout longer.

Thanks for your efforts!

Add a sampler filter in the configuration

Is your feature request related to a problem? Please describe.
Since we use protocol based testing with JMeter we have a lot of resources as test steps in our test plan. But only a few a necessary to report to Dynatrace.

Describe the solution you'd like
We would like to have the possibility to filter sampler names by regex as the InfluxDBListener does.

Dynatrace API Endpoint and Token Sanity Check before execution

When passing in a wrong or invalid API Token or Dynatrace API Endpoint the plugin tries to call that invalid combination everytime it tries to send data. This results in a lot of error log outputs during a JMeter test execution that uses this plugin

I suggest that the plugin should do an initial check on whether the API is reachable using the passed Token. If that is not the case the plugin should create a single error log but then stop sending data as this data would never reach Dynatrace anyway

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.