Code Monkey home page Code Monkey logo

node-statsd-client's Introduction

DEPRECATED node-statsd-client

This module is no longer maintained - please use alternative libraries, such as hot-shots.

Node.js client for statsd.

Build Status

Quick tour

var SDC = require('statsd-client'),
	sdc = new SDC({host: 'statsd.example.com'});

var timer = new Date();
sdc.increment('some.counter'); // Increment by one.
sdc.gauge('some.gauge', 10); // Set gauge to 10
sdc.timing('some.timer', timer); // Calculates time diff
sdc.histogram('some.histogram', 10, {foo: 'bar'}) // Histogram with tags
sdc.distribution('some.distribution', 10, {foo: 'bar'}) // Distribution with tags

sdc.close(); // Optional - stop NOW

API

Initialization

var SDC = require('statsd-client'),
	sdc = new SDC({host: 'statsd.example.com', port: 8124});

Global options:

  • prefix: Prefix all stats with this value (default "").
  • tcp: User specifically wants to use tcp (default false).
  • socketTimeout: Dual-use timer. Will flush metrics every interval. For UDP, it auto-closes the socket after this long without activity (default 1000 ms; 0 disables this). For TCP, it auto-closes the socket after socketTimeoutsToClose number of timeouts have elapsed without activity.
  • tags: Object of string key/value pairs which will be appended on to all StatsD payloads (excluding raw payloads) (default {})

UDP options:

  • host: Where to send the stats (default localhost).
  • port: Port to contact the statsd-daemon on (default 8125).
  • ipv6: Use IPv6 instead of IPv4 (default false).

TCP options:

  • host: Where to send the stats (default localhost).
  • port: Port to contact the statsd-daemon on (default 8125).
  • socketTimeoutsToClose: Number of timeouts in which the socket auto-closes if it has been inactive. (default 10; 1 to auto-close after a single timeout).

HTTP options:

  • host: The URL to send metrics to (default: http://localhost).
  • headers: Additional headers to send (default {})
  • method: What HTTP method to use (default PUT)

To debug, set the environment variable NODE_DEBUG=statsd-client when running your program.

Counting stuff

Counters are supported, both as raw .counter(metric, delta) and with the shortcuts .increment(metric, [delta=1]) and .decrement(metric, [delta=-1]):

sdc.increment('systemname.subsystem.value'); // Increment by one
sdc.decrement('systemname.subsystem.value', -10); // Decrement by 10
sdc.counter('systemname.subsystem.value', 100); // Increment by 100

Gauges

Sends an arbitrary number to the back-end:

sdc.gauge('what.you.gauge', 100);
sdc.gaugeDelta('what.you.gauge', 20);  // Will now count 120
sdc.gaugeDelta('what.you.gauge', -70); // Will now count 50
sdc.gauge('what.you.gauge', 10);       // Will now count 10

Sets

Send unique occurences of events between flushes to the back-end:

sdc.set('your.set', 200);

Delays

Keep track of how fast (or slow) your stuff is:

var start = new Date();
setTimeout(function () {
	sdc.timing('random.timeout', start);
}, 100 * Math.random());

If it is given a Date, it will calculate the difference, and anything else will be passed straight through.

And don't let the name (or nifty interface) fool you - it can measure any kind of number, where you want to see the distribution (content lengths, list items, query sizes, ...)

Histogram

Many implementations (though not the official one from Etsy) support histograms as an alias/alternative for timers. So aside from the fancy bits with handling dates, this is much the same as .timing().

Distribution

Datadog's specific implementation supports another alternative to timers/histograms, called the distribution metric type. From the client's perspective, this is pretty much an alias to histograms and can be used via .distribution().

Raw

Passes a raw string to the underlying socket. Useful for dealing with custom statsd-extensions in a pinch.

sdc.raw('foo.bar:123|t|@0.5|#key:value');

Tags

All the methods above support metric level tags as their last argument. Just like global tags, the format for metric tags is an object of string key/value pairs. Tags at the metric level overwrite global tags with the same key.

sdc.gauge('gauge.with.tags', 100, {foo: 'bar'});

Express helper

There's also a helper for measuring stuff in Express.js via middleware:

var SDC = require('statsd-client');
var app = express();
	sdc = new SDC({...});

app.use(sdc.helpers.getExpressMiddleware('somePrefix'));
// or
app.get('/',
	sdc.helpers.getExpressMiddleware('otherPrefix'),
	function (req, res, next) { req.pipe(res); });

app.listen(3000);

This will count responses by status-code (prefix.<statuscode>) and the overall response-times.

It can also measure per-URL (e.g. PUT to /:user/:thing will become PUT_user_thing by setting the timeByUrl: true in the options-object:

app.use(sdc.helpers.getExpressMiddleware('prefix', { timeByUrl: true }));

As the names can become rather odd in corner-cases (esp. regexes and non-REST interfaces), you can specify another value by setting res.locals.statsdUrlKey at a later point.

The / page will appear as root (e.g. GET_root) in metrics while any not found route will appear as {METHOD}_unknown_express_route. You can change that name by setting the notFoundRouteName in the middleware options.

Callback helper

There's also a helper for measuring stuff with regards to a callback:

var SDC = requrire('statsd-client');
	sdc = new SDC({...});

function doSomethingAsync(arg, callback) {
	callback = sdc.helpers.wrapCallback('somePrefix', callback);
	// ... do something ...
	return callback(null);
}

The callback is overwritten with a shimmed version that counts the number of errors (prefix.err) and successes (prefix.ok) and the time of execution of the function (prefix.time). You invoked the shimmed callback exactly the same way as though there was no shim at all. Yes, you get metrics for your function in a single line of code.

Note that the start of execution time is marked as soon as you invoke sdc.helpers.wrapCallback().

You can also provide more options:

sdc.helpers.wrapCallback('somePrefix', callback, {
	tags: { foo: 'bar' }
});

Stopping gracefully

By default, the socket is closed if it hasn't been used for a second (see socketTimeout in the init-options), but it can also be force-closed with .close():

var start = new Date();
setTimeout(function () {
	sdc.timing('random.timeout', start); // 2 - implicitly re-creates socket.
	sdc.close(); // 3 - Closes socket after last use.
}, 100 * Math.random());
sdc.close(); // 1 - Closes socket early.

The call is idempotent, so you can call it "just to be sure". And if you submit new metrics later, the socket will automatically be re-created, and a new timeout-timer started.

Prefix-magic

The library supports getting "child" clients with extra prefixes, to help with making sane name-spacing in apps:

// Create generic client
var sdc = new StatsDClient({host: 'statsd.example.com', prefix: 'systemname'});
sdc.increment('foo'); // Increments 'systemname.foo'
... do great stuff ...

// Subsystem A
var sdcA = sdc.getChildClient('a');
sdcA.increment('foo'); // Increments 'systemname.a.foo'

// Subsystem B
var sdcB = sdc.getChildClient('b');
sdcB.increment('foo'); // Increments 'systemname.b.foo'

Internally, they all use the same socket, so calling .close() on any of them will allow the entire program to stop gracefully.

What's broken

Check the GitHub issues.

Other resources

  • statsd-tail - A simple program to grab statsd-data on localhost
  • hot-shots - Another popular statsd client for Node.js
  • statsd - The canonical server

RELEASES

See the changelog.

LICENSE

ISC - see LICENSE.

node-statsd-client's People

Contributors

addisonj avatar bolgovr avatar cactusbone avatar chris911 avatar clarkie avatar dependabot[bot] avatar dlebech avatar dougnukem avatar garthk avatar gochomugo avatar jbt avatar jergason avatar jgantunes avatar kevinhuang12345678 avatar msiebuhr avatar olemchls avatar papandreou avatar philippeg-playster avatar quinox avatar sdepold avatar sirzach avatar skellla avatar theconnman avatar thiagocaiubi avatar tlhunter avatar vekexasia avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-statsd-client's Issues

Tests fail on OS X by default

When running just npm test, the StatsDClient-tests fail. But when the Ephemeral-socket tests are skipped, the very same tests work fine.

I suspect I've messed up something in either EphemeralSocket-tests or FakeServer.

Sanitize setup

In complex setups, having mostly-working defaults all over makes debugging quite hard.

  • It should be in-active when no configuration is given (at least no host is given).
  • Some kind of logging. Consider giving a console-like object when setting up, to do some logging.
  • Assert-guards to make sure stuff is set up correctly.

Anything else?

StatsD client & graphite on different servers

Hi

I have installed nodejs - statsd Client on server 1. Graphite installed on Server 2

What are the changes required to display server1 statsd UDP points on Server 2 graphs ??

Well this is not an issue. Its a query. Please suggest..

wait for socket until everything has been sent

Hi,

please have a look at the following code:

run();
async function run() {
    //..... doing some async / await stuff here .....//

    const sdc = new SDC({ host: appConfig.statsdHost, prefix: `test` });
    sdc.increment("testcounter", 1);

    sdc.close();
    process.exit();
};

This will cause that stats will not be sent to the server. Is there any way to await for the socket until everything is done?

Tag format incompatible with Telegraf

Hi,
I'm using this library to send data to telegraf-statsd server. All seems to work fine except when I attach tags to the metrics.

Given this sentence sdc.increment('my_counter', 2, { service: 'value' }); the library send the message my_counter:2|c|#service:value while telegraf-statsd expects something like my_counter,service=value:+3|c.

Does the library supports telegraf?

set() doesn't track uniques

What I expected:

statsd.set('names', 'A');
statsd.set('names', 'B');
statsd.set('names', 'C');

… with names:3|s emitted.

Instead, I need to statsd.set('names', 3)… in which case, keeping track of the set and when to emit is left to me.

Benchmarks

This is all about getting data out the door. Let's make sure we don't have any obvious performance issues around.

Ex. we bind() the UDP-socket for no apparent reason, which might not be a good idea...

  • Find some benchmarking library we can use
  • Write some basic benchmarks

Test the Express-helper

The express-helper is quickly becoming a somewhat complex part of the code-base, and is completely devoid of testing. (But, as we've found out, not bugs...)

Test-suite

Both with the project as-is, and the ideas I (and, probably, others) have for new features, there really should be a test-suite.

Ideally, a small faux statsd-server would be ideal, but other ideas are welcome too.

Are there any ways to make sure statsdclient is working?

I'm expecting statsclient could emit some events that would help developers to observer its status.

E.g:

sdc.on('connect', (): void=>{});
sdc.on('data', (): void=>{});
sdc.on('error', (): void=> {});
sdc.on('close', (): void=>{});

upgrade to use node 14, replace deprecated new Buffer() by Buffer.from()

workflow-regression_1 | TypeError: Cannot convert a Symbol value to a string
workflow-regression_1 | at Socket. (node_modules/zone.js/dist/zone-node.js:1665:52)
workflow-regression_1 | at enqueue (dgram.js:498:10)
workflow-regression_1 | at Socket.send (dgram.js:633:5)
workflow-regression_1 | at /opt/haven/app/node_modules/statsd-client/lib/EphemeralSocket.js:182:22
workflow-regression_1 | at EphemeralSocket._createSocket (node_modules/statsd-client/lib/EphemeralSocket.js:106:16)
workflow-regression_1 | at EphemeralSocket._send (node_modules/statsd-client/lib/EphemeralSocket.js:174:10)
workflow-regression_1 | at EphemeralSocket._flushBuffer (node_modules/statsd-client/lib/EphemeralSocket.js:150:10)
workflow-regression_1 | at EphemeralSocket._enqueue (node_modules/statsd-client/lib/EphemeralSocket.js:138:14)
workflow-regression_1 | at EphemeralSocket.send (node_modules/statsd-client/lib/EphemeralSocket.js:161:14)
workflow-regression_1 | at StatsDClient.counter (node_modules/statsd-client/lib/statsd-client.js:82:18)
workflow-regression_1 | at StatsDClient.increment (node_modules/statsd-client/lib/statsd-client.js:91:10)

suggest code change, from:
EphemeralSocket.prototype._send = function _send(data) {
debug("_send(", data, ")");
// If we don't have a socket, or we have created one but it isn't
// ready yet, we need to enqueue data to send once the socket is ready.
var that = this;

this._createSocket(function () {
    that._socketUsed = true;

    // Create message
    var message = new Buffer(data);

    debug(message.toString());

    that._socket.send(message, 0, message.length, that._port, that._hostname);
});

};

to:

EphemeralSocket.prototype._send = function _send(data) {
debug("_send(", data, ")");
// If we don't have a socket, or we have created one but it isn't
// ready yet, we need to enqueue data to send once the socket is ready.
var that = this;

this._createSocket(function () {
    that._socketUsed = true;

    // Create message
    var message = Buffer.from(data);

    debug(message.toString());

    that._socket.send(message.toString(), 0, message.length, that._port, that._hostname);
});

};

Quick tour in README.md does not work

This code from the readme:

var sdc = new require('statsd-client')({host: 'statsd.example.com'});

var timer = new Date();
sdc.increment('some.counter'); // Increment by one.
sdc.gauge('some.gauge', 10); // Set gauge to 10
sdc.timing('some.timer', timer); // Calculates time diff

sdc.close(); // Optional - stop NOW

doesn't work:

$ node test.js 

test.js:4
sdc.increment('some.counter'); // Increment by one.
    ^
TypeError: Cannot call method 'increment' of undefined
    at Object.<anonymous> (/Users/josephg/src/livedb/test.js:4:5)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3

The problem is this new require('statsd-client')({host: 'statsd.example.com'}); is actually interpretted as (new require('statsd-client'))({host: 'statsd.example.com'}); or something like that, and it returns undefined.

This works:

var SDC = require('statsd-client');
var sdc = new SDC({host: 'localhost'});

var timer = new Date();
sdc.increment('some.counter'); // Increment by one.
sdc.gauge('some.gauge', 10); // Set gauge to 10
sdc.timing('some.timer', timer); // Calculates time diff

sdc.close(); // Optional - stop NOW

0.15 missing commit

For whatever reason, the 0.15 module currently on NPM does not have the fix from commit 2273d5a. This prevents the node process from terminating:

var Client = require('statsd-client');
var stats = new Client({
  host: '',
  port: null

});
stats._ephemeralSocket.log = console.log;

stats.increment('blah');
stats.close();

TypeError: Cannot call method 'send' of null

I am receiving this error intermittently. Seems to happen most frequently when a request takes a long time. I believe the bug is in lib/statsd-client.js lines 117-126.

StatsDClient.prototype._send = function (data) {
    // Create socket if it isn't there
    if (!this._socket) {
        this._socket = dgram.createSocket('udp4');

        // Start timer, if we have a positive timeout
        if (this.options.socket_timeout > 0) {
            this._socket_timer = setTimeout(this._socket_timeout.bind(this), this.options.socket_timeout);
        }
    }

It seems like you are expecting code within the conditional to block, but when it doesn't it reaches the 'send' and then you receieve the error 'cannot call method send of null'

More flexible prefix-support

We need to collect statistics in several corners of our app; we should have some kind of support for managing this a lá namespacing.

Middleware with timeByUrl path is not being parsed correctly

Debug output from statsd server:

1 Nov 14:12:46 - DEBUG: prefix.[object Object].response_time:3132|ms

This is how I'm using the module:

'use strict';

const SDC = require('statsd-client');
const sdc = new SDC({debug: true, prefix: 'prefix'});

sdc.increment('working');

const metricsMiddleware = sdc.helpers.getExpressMiddleware({
  timeByUrl: true
});

  const app = express();
  app.disable('x-powered-by');
  app.use(cors);
  app.use(bodyParser.json());
  app.use(bodyParser.urlencoded({extended: true}));
  app.use(metricsMiddleware);

Callback-timing helper

From NewRelic's Node API:

asyncFuncToMeasure(some, arguments, statsd.timeAsyncFunc("foo.bar.baz", callback(...))

It should basically start the timer when it is initially called wrap the given function in a small shim that will submit the time taken to execute.

Remove stream-helpers

Streams will change significantly in Node 0.10, so these will be worthless as-is.

And does anybody actually use the stream-helpers?

Can't post to StatsD with 0.0.6.

With a fresh install from npm, I can't get the client to post to StatsD server.

 npm install statsd-client

Slightly modified example to post to StatsD server running on localhost. Result - no stats get posted.

Adding debug code to EphemeralSocket.js, it's definitely trying to send. Server is running on default port 8125 (and I tried another statsd client that is posting, so I know server is working).

var sdc = require('statsd-client'),
SDC = new sdc({ debug: 1, prefix: "statsd-client"}),
SDCTest = SDC.getChildClient('test');

var begin = new Date();
setTimeout(function () {
    // Set 'statsd-client.test.gauge'
    SDCTest.gauge('gauge', 100 * Math.random());

    // Icrement 'statsd-client.test.counter' twice
    SDCTest.increment('counter');
    SDC.increment('test.counter');

    // Set some time
    SDC.timing('speed', begin);

    // Close socket
    SDC.close();
}, 100 * Math.random());

Measure on streams

Create a handler of sort, that makes it easy to measure what's going on in streams;

var measureStream = sdc.getStreamMetric('key.to.report'); // Return readable+writable stream
input.pipe(measureStream).pipe(output)

This would then report how much data goes from input to output via the counter key.to.report.

An advanced version would allow one to do custom stuff on stream events; eg data, end, error and so on:

var mS = sdc.getStreamMetric('key.to.report', {
    data: function (sdc, data) { sdc.increment('data', data.length); },
    end: function (sdc) {sdc.increment('end'); }
});
...

Which would count all passing data in key.to.report.data and how many connections that end in key.to.report.end.

Explicit .close() for the socket

When batch-programs are done, it would be nicer to call a .close() to close the socket and kill the timer, rather than having to wait ~1½ cycles of the timer for the program to end.

Support name prefixes

It would be nice to have support for an automatically added prefix, eg. to put appname.subsystem in front of all sent metrics.

StatsD protocol support

There are regular PR's to support different variants of the statsd protocol:

  • InfluxDB: #74
  • "Datadog-style" (#69) which ex. Prometheus also seem to support. Has some slightly different names from "standard" statsd.
  • Telegraf (#76), which apparently requires sample-rates on some types.

It's all getting quite fragmented, and we can't possibly support all of it without having some kind of strategy.

So: What servers are popular out there (i.e. which should we support) and any ideas besides throwing more arguments on the functions we have?

statsdUrlKey

Hi,

I'm using statsd-client in an express application which exposes a REST API with several endpoints (URLs).

I use statsdUrlKey to distinguish between endpoints and it works great with response times.

Is there any reason why statsdUrlKey is ignored when pushing response_code metrics to statsd?
It could be useful to distinguish the route names for response codes also. Example: compute and display the number of requests for a given endpoint and not the total for all endpoints ...

Thanks for your (future) answer.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.