Code Monkey home page Code Monkey logo

nexogen.libraries.metrics's Introduction

Nexogen.Libraries.Metrics

Library for collecting application metrics in .Net and exporting them to Prometheus

Build Status Build status GitHub license NuGet NuGet GitHub issues GitHub stars

API Reference

Nexogen.Libraries.Metrics API Reference

Updating from version 2.6.0 or earlier

The default Prometheus metrics registration behavior in ASP.NET Core applications has changed. The request metrics collection is no longer enabled by default. This means that you need to add code to enable it. See the relevant section below for code samples. Explicitly defined metrics are not affected in any way.

Installation

dotnet add package Nexogen.Libraries.Metrics.Prometheus
dotnet add package Nexogen.Libraries.Metrics.Extensions

You can use an interface only nuget when writing libraries or when you want to use Metrics through dependency injection.

dotnet add package Nexogen.Libraries.Metrics

For exporting metrics you currently have to use ASP.NET Core.

dotnet add package Nexogen.Libraries.Metrics.Prometheus.AspCore

Or you can use a push gateway when measuring batch processes.

dotnet add package Nexogen.Libraries.Metrics.Prometheus.PushGateway

Example usage - Measurements

Counters

Counters can only increment, so they are most useful for counting things, like calls to API endpoints or backend services.

IMetrics metrics = new PrometheusMetrics();

ICounter counter = metrics.Counter()
    .Name("nexogen_sort_calls_total")
    .Help("Total calls to sort routine.")
    .Register();

counter.Increment();

Gauges

Gauges can take any value, so they are the most versatile metric type available. You can even measure durations or dates with them!

IGauge gauge = metrics.Gauge()
    .Name("nexogen_sorted_items_count_last")
    .Help("The last count of the sorted items.")
    .Register();

gauge.Value = items.Length;

gauge.Increment();
gauge.Decrement(10.1);           

Histograms

Histograms are a trade off between measuring resolution and precision. With histograms you can avoid aliasing errors from Prometheus's scrape interval, but lose granularity. Histograms also need to have their buckets defined before use and we provide sevaral bucket generators to make it easy.

IHistogram histogram = metrics.Histogram()
    .LinearBuckets(0.01, 0.01, 100)
    .Name("nexogen_sort_time_seconds")
    .Help("Time taken for sort in seconds.")
    .Register();

var sw = Stopwatch.StartNew();
Array.Sort(items);
histogram.Observe(sw.Elapsed.TotalSeconds);

Extensions

We provide an Extensions library for making common measurements easy.

using (histogram.Timer())
{
    Array.Sort(items);
}

gauge.SetToCurrentTime();

gauge.TrackInProgress(() => Array.Sort(items));

ASP.NET Core metrics collection

There is a way to automatically collect metrics about the ASP.NET Core requests. This needs to be enabled when registering to the application.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UsePrometheus(options => options.CollectHttpMetrics());
}

The options.CollectHttpMetrics() registers automatic request metric collection, which was implicitly registered in versions prior to version 3.0.0.

Example usage - Exposing metrics

Asp.Net Core

The collected metrics can be exposed via ASP.NET Core middleware. To do this you need to register the feature during configuration.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UsePrometheus();
}

Standalone server

There is a standalone server if you don't want to use ASP.NET Core just to expose your metrics.

dotnet add package Nexogen.Libraries.Metrics.Prometheus.Standalone
public void ConfigureServices(IServiceCollection services)
{
    services.AddPrometheusStandalone(Configuration.GetSection("Prometheus"));
}

or

var metrics = new PrometheusMetrics();
await new PrometheusServer(new PrometheusServerOptions {Port = 9100}, metrics, loggerFactory).StartAsync();

gRPC

We provide gRPC interceptors for capturing metrics.

dotnet add package Nexogen.Libraries.Metrics.Prometheus.Grpc

For gRPC servers:

services.AddSingleton<IGrpcServerMetrics, GrpcServerMetrics>()
        .AddGrpc(options => options.Interceptors.Add<ServerMetricsInterceptor>());

For gRPC clients:

services.AddSingleton<IGrpcClientMetrics, GrpcClientMetrics>()
        .AddGrpcClient<T>((provider, options) => options.Interceptors.Add(new ClientMetricsInterceptor(provider.GetRequiredService<IGrpcClientMetrics>())));

nexogen.libraries.metrics's People

Contributors

ahoka avatar bastianeicher avatar discosultan avatar endofcake avatar gebolze avatar kodfodrasz avatar mihalykoantal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

nexogen.libraries.metrics's Issues

.net framework nuget library missing

.NET Framework library is missing in NuGet package. Any particular reason for that?
I mean the .NET Standard is cool and all that, but it would drag a ton of references to a .NET Framework project.

Add an option to specify default label values

The way it works currently is that you define label names statically at metric creation time:

            var planningTime = metrics.Gauge()
                .Name("planning_time").Help("The processing time in seconds").LabelNames("solver").Register();

And then you have to specify label values every time you do anything with this metric:

planningCount.Labels(solver).Increment();

There's no way to set label values once, which is a bit of a pain for things that don't change within the lifetime of a given instance (for example, environment or instance ID).

Allow using CollectionMiddleware without exposing metrics endpoint

I would like to have an option to add CollectMetricsMiddleware without being forced to have metrics are being exposed on /metrics path in AspNetCore.

Would prefer to use StandaloneService or my own on another port for security reasons.

Making CollectMetricsMiddleware / HttpMetrics classes public and simple helper for middleware registration would be enough.

Do you accept PRs?

Label values are not escaped

Based on the Prometheus documentation: https://prometheus.io/docs/instrumenting/exposition_formats/#text-format-details

metric_name and label_name have the usual Prometheus expression language restrictions. label_value can be any sequence of UTF-8 characters, but the backslash, the double-quote, and the line-feed characters have to be escaped as \, ", and \n, respectively

Actual Output:

tokens_login{username="domain\username"} 1 1510911080610

Expected Output:

tokens_login{username="domain\\username"} 1 1510911080610

Provide a way to create IMetrics outside of ```AddPrometheus()``` extension method

I'm building a background service that I wanted to collect metrics from the background service & expose them using http endpoint. Unfortunately there is no easy way to share the metrics unless you resolve metrics from the service provider, can we provide a method that has an optional metrics parameter?

 AddPrometheus(this IServiceCollection services, IMetrics metrics = null)
{
    metrics = metrics ?? new PrometheusMetrics();
    //rest of code
}

Nexogen.Libraries.Metrics.Prometheus.Standalone package missing

The readme says to install Nexogen.Libraries.Metrics.Prometheus.Standalone for a standalone server but there is no release of this package for 3.1.0 (the last is 3.1.0-rc.1).

Commit 5c4e3cc seems to have removed the standalone server code. Does this mean that this feature is no longer supported?

Perhaps it could be reimplemented using .NET Core Hosted Services and HttpListener, for example like this:

public class PrometheusServer : IHostedService
{
    private readonly HttpListener _listener;
    private readonly IExposable _metrics;

    public PrometheusServer(IOptions<PrometheusServerOptions> options, IExposable metrics)
    {
        _listener = new HttpListener {Prefixes = {$"http://*:{options.Value.Port}/"}};
        _metrics = metrics;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        _listener.Start();
        BeginContext();
        return Task.CompletedTask;
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        _listener.Abort();
        return Task.CompletedTask;
    }

    private void BeginContext()
    {
        _listener.BeginGetContext(ListenerCallback, _listener);
    }

    private async void ListenerCallback(IAsyncResult result)
    {
        var context = _listener.EndGetContext(result);
        context.Response.StatusCode = 200;
        context.Response.Headers.Add("Content-Type", "text/plain");
        await _metrics.Expose(context.Response.OutputStream, ExposeOptions.Default);
        context.Response.Close();
    }
}

I could create a Pull Request if you're interested.

Remove deprecated metric with a specific label.

First of all, thank you for your work!

It is a great feature to be able to create new metrics by just using a new label. However that would be great to be able to remove metrics with a specific deprecated label. Please see the following example.

this.Something = metrics.Gauge()
    .Name("something_items_total")
    .Help("Total number of items")
    .LabelNames("someLabel")
    .Register();
# HELP something_items_total Total number of items
# TYPE something_items_total gauge
something_items_total{someLabel="label1"} 5 1513340498878
something_items_total{someLabel="label2"} 2 1513340498878
...

At some point items labeled with "label1" becomes obsolete and therefore its metric.. I can reset it to 0, but cannot remove it :(

This is what I would like to see:

# HELP something_items_total Total number of items
# TYPE something_items_total gauge
something_items_total{someLabel="label2"} 2 1513340498878
...

Correct me if I am wrong and it is a misuse of metrics, however if I do not remove it, it becomes kind of a tiny memory leak :)

Attaching `instance` to PushGateway payload

Hello.
Great library, by the way, thanks for publishing it.
I'm trying to start using PushGateway functionality for our metrics, and at the moment it doesn't look like it's possible to identify where a given metric is coming from.

Here's how the path is configured:

var path = String.Format("/metrics/job/{0}", WebUtility.UrlEncode(job));

https://github.com/nexogen-international/Nexogen.Libraries.Metrics/blob/master/Nexogen.Libraries.Metrics.Prometheus.PushGateway/PushGateway.cs#L115

My understanding is that if it were in the form

var path = String.Format("/metrics/job/{0}/instance/{1}", WebUtility.UrlEncode(job), WebUtility.UrlEncode(instance));

it would be possible to identify the instance where the metric is coming from.

I see there's a way to set custom metrics, but having an option to set instance would still be handy.

ASP.Net Core endpoint response time collecting middleware uses actual path as labels

Nexogen.Libraries.Metrics.Prometheus.AspCore.HttpMetricsMiddleware measures endpoint response times and collects them labelled by http methods, response codes and endpoint paths.

However these paths are actual paths (eg.: order/321item/123), not logical ones (eg.: order/{orderId}/item/{itemId}) identifiing the controller in MVC, thus if [PathParam] is used on a controller this causes an explosion of labels by each actual call. This has storage and performance costs in Prometheus.

The middleware level approach currently used doesn't have acces to MVC routing, so the logical path is not available, only actual HTTP request path.

This is usually not desired, so this measurement should either be moved to MVC level, or an MVC level alternative needs to be added, so the consumers can weigh the costs and benefits of each approach.

Issue with CoreCLR metric

I am pushing a set of Gauges that I recorded but when scraping the metrics in my push gateway I get the following error:

expected counter in metric process_cpu_seconds_total label:<name:"instance" value:"" > label:<name:"job" value:"test_job" > label:<name:"test_id" value:"978e7cdb-c017-4e69-a9ac-719f96deee91" > gauge:<value:0.9375 >

I am not capturing nor sending this (actively) so it seems like the set up of the metric in the coreclrexporter class is not done properly.

.net framework compatibility

i'm trying to run this on .net 4.6.1 and getting this error:

System.MissingMethodException occurred
  HResult=0x80131513
  Message=Method not found: 'System.Collections.Generic.IEnumerable`1<!!0> System.Linq.Enumerable.Prepend(System.Collections.Generic.IEnumerable`1<!!0>, !!0)'.
  Source=Nexogen.Libraries.Metrics.Extensions
  StackTrace:
   at Nexogen.Libraries.Metrics.Extensions.Buckets.BucketGenerator.Buckets(Double[] bounds)
   at Nexogen.Libraries.Metrics.BucketExtensions.Buckets(IHistogramBuilder builder, Double[] bounds)
   at Nexogen.Libraries.Metrics.Prometheus.AspCore.HttpMetrics..ctor(IMetrics m)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass16_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
   at Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass3_0.<UseMiddleware>b__0(RequestDelegate next)
   at Microsoft.AspNetCore.Builder.Internal.ApplicationBuilder.Build()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at OrleansDashboard.Dashboard.<Init>d__15.MoveNext() in C:\Users\tal\dev\OrleansDashboard\OrleansDashboard\Dashboard.cs:line 99

i see you have a compat library, but it only adds the required extensions in .net 4.5.2

this needs to be done all the way to .net 4.7.x

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.