nodejitsu / godot Goto Github PK

What has always been my favorite part of godot is its use of streams. Transform, "read-write" or "through" streams are the best mechanism for assessing data on the fly and we use them heavily in godot for all our reactors. Currently we are able to setup a reactor pipe-chain as follows..

var g = require('godot');

var pipeChain = 
  g.reactor()
    .where('service', '*/health/heartbeat')
    .expire(1000 * 60)
    .email({ to: '[email protected]' })

Now even though this provides a simple and clean api, it hides the fact that we are using native node streams and also restricts users to the transform streams that we have available as reactors. We want to expose the raw unifying interface that native streams provide to allow you to have greater control over your event processing!

With this goal in mind, we have a couple api possibilities based on my discussion with @indexzero. Both will allow the use of the native node .pipe() method for creating custom pipe chains, allowing you to use any object based transform stream available in npm!

The other change that you will notice is that we will expose all of the native godot reactor streams on the godot object itself.

var g = require('godot');

//
// Reactor server which will email `[email protected]`
// whenever any service matching /.*\/health\/heartbeat/
// fails to check in after 60 seconds.
//
g.createServer({
  //
  // Defaults to UDP
  //
  type: 'udp',
  reactors: [
    g.where('service', '*/health/heartbeat')
     .pipe(g.expire(1000 * 60))
     .pipe(g.email({ to: '[email protected]' }))
  ]
}).listen(1337);

So the first api is the most ideal IMO. All you need to do is pass your own custom pipe chain(s) into the reactors array and voila!

The other API is as follows..

var g = require('godot');

//
// Reactor server which will email `[email protected]`
// whenever any service matching /.*\/health\/heartbeat/
// fails to check in after 60 seconds.
//
g.createServer({
  //
  // Defaults to UDP
  //
  type: 'udp',
  reactors: [
    function health(socket) {
      socket.
       .pipe(g.where('service', '*/health/heartbeat'))
       .pipe(g.expire(1000 * 60))
       .pipe(g.email({ to: '[email protected]' }));
    }
  ]
}).listen(1337);

Now even though I like the first API better, It may not be possible to preserve the multiplexing functionality of godot where you are able to create a separate stream for each incoming connection. This API should ensure this is possible.

Now this is just the beginning of the road to godot v1.0. Please post any questions or concerns if you have any!

Stores events in Redis.

Just want to try a little bit on godot and follow readme guideline, but it will always throw error during createServer?

/home/ferrari/Programming/nodejs/nodejs-experiment/node_modules/godot/lib/godot/net/server.js:148
throw err;
^
undefined

https://github.com/nodejitsu/godot/blob/master/lib/godot/net/server.js#L134-L150
When it create server and listen, will go into this function, but responded is false and it will always throw error in the end, is this correct? Or just I use the wrong way to create godot server? thanks

Emits the difference of the number of events between two keys on given data objects.

npm test currently fails because:

  test/net/client-test.js
  test/net/server-test.js

are tests failing with TODO! Write this test.

How do I pass sendgrid auth details ?

regards

Hello,
first of all I wish to thank you for this great module: I have been using it since its publication.
Unfortunately I have the following issue while closing a udp client: it seems that producer intervals are not cleared preventing server closing.
I am creating a new client and it's producer as per code below:

var godotProducer = godot.producer({
        host: '127.0.0.1',
        service: 'heartbeat',
        ttl: 1000 * 15,
}),

var godotClient = godot.createClient({
        type: 'udp',        
});

godotClient.add(godotProducer); 

godotClient.connect(10002); 

Then, if I do not run the following clearInterval instruction before closing godot's client connection, my tcp server won't close:

clearInterval(godotProducer.ttlId);

godotClient.close();

Many thanks in advance

See: http://riemann.io/api/riemann.streams.html#var-by

What reimann considers a just a clojure function is an instantiatable pipe-chain in godot (i.e. godot.reactor()...createStream())

Your project has a name conflict with https://github.com/godotengine/godot

Right now something like:

  var godot = require('godot');

  godot.createServer({
    reactors: [
      godot.reactor()
        .change('state')
    ]
  });

Can only listen when the state property on data changes, but not when it changes from one specific value to another

Ran into this with the by reactor. It sends the warning when it is piping to new reactor streams as that adds a data listener for each pipe that is called.

We are close to a 1.0 for godot. These issues need to be addressed and 1.0.0 will only work on node > 0.10.0 because of streams2

Most of the math is already there:

https://github.com/nodejitsu/godot/blob/master/lib/godot/common/ewma.js
https://github.com/nodejitsu/godot/blob/master/lib/godot/common/window.js

Need binaries for something like:

  godot-server --config /path/to/godot-config-file
  godot-client --config /path/to/godot-config-file

Hi there,
I've been trying some tests with Godot, while it's really handy to get it working with the standard examples (heartbeat case), I could not find any mention of customized events sent by a Godot producer.

Moar docs branch is helpful but lacks references in this domain, whereas the godot introducing entry on Nodejistu mention the ('service', 'cpu').

Could you please paste here some quick draft example ?
I could then try to fill the documentation, if you guys do not have time for now.

Many thanks

No tests for this. Unsure the best way to ensure delivery in tests. Could mock, but that seems counter-productive.

Things to look at:

• https://github.com/dominictarr/reconnect
• https://github.com/MathieuTurcotte/node-backoff

Currently the setting is to drop any messages that are written to the client before it connects. This currently works for my use cases but it could be advantageous to buffer these messages to be sent in batch once the client does connect.

Thoughts?

This should be emitted internally by reactors. This will solve #57 and provide enable the error event to signify fatal errors.

Another Riemann feature that seems sane / useful:

rollup will allow a few events to pass through readily. Then it starts to accumulate events,
rolling them up into a list which is submitted at the end of a given time interval.

See Rieman Docs for more info

Rollup timeout should be configurable based on the interval number.

e.g.

• Rollup 10,000 events every 5 minutes for an hour.
• After that Rollup 10,000 events every 30min from then on.

So in the simple case the error propagation works as expected as shown in the map error tests and the macro that does the heavy lifting.

Now I believe the core problem exists with how we set up the reactor pipe-chain but I believe that will be a problem assessed in bringing godot to v1.0.0.

Anyways, somehow when there is an influx of errors being emitted from a reactor primitive, the stream is stalled and can no longer accept anymore events. I think the best approach to take is to do some dtrace testing on how the bottleneck is created underneath the hood.

How its supposed to behave is the error handlers here are placed to propagate the reactor's errors to the reactor instance to then be handled by the server instance.

Did some live testing for #55 and found the blocking issue I think. The module itself is throwing an error when attempting to run a backoff. Easiest solution is to replace this with a simpler module back.

/node_modules/godot/node_modules/backoff/lib/backoff.js:50
        throw new Error('Backoff in progress.');
              ^
Error: Backoff in progress.
    at Backoff.backoff (/node_modules/godot/node_modules/backoff/lib/backoff.js:50:15)
    at Socket.onError (/node_modules/godot/lib/godot/net/client.js:177:44)
    at Socket.EventEmitter.emit (events.js:95:17)
    at onwriteError (_stream_writable.js:233:10)
    at onwrite (_stream_writable.js:251:5)
    at WritableState.onwrite (_stream_writable.js:97:5)
    at fireErrorCallbacks (net.js:438:13)
    at Socket._destroy (net.js:449:5)
    at Socket._write (net.js:635:10)
    at doWrite (_stream_writable.js:221:10)

See: https://github.com/nodejitsu/godot/blob/master/lib/godot/producer/producer.js#L25

Argument type checking always expects a port even though we say it's optional. Workaround:

client.connect(port, callback);

godot emits \n delimited JSON. There are so many streaming JSON parsers out there I'm unsure which one actually handles this.

Would like to see a list of actual test cases before we dig into writing them more:

Right now the one that really needs to be implemented is:

• Expired TTL.

Hey guys i have seen that the last commit was basically 6 years ago but i am curios if there is at least someone who takes care of some issues or maybe maintaining the project ?

The blog post indicates this project is available under the MIT license. However, I can't find any license information in the code itself, other than Copyright Nodejitsu Inc.

Could you add an explicit LICENSE file to the repo?

Riemann has a short-hand stream processor called "Tagged". This is like .where() except that it only checks to see if the event has a given tag.

We are going to need to do a non-trivial refactor for streams2 when it's ready, but we should not wait for [email protected]

This is a cross-cutting concern across most Reactor primitives. When things start to go wrong it is likely for a lot of reactors to fire at once. This is likely to be a flood of email and SMS

So debouncing would only send these messages after a certain amount of time has passed.

Seems pretty necessary if we want to send fancy emails to people when there app goes down.

Anyone wants this issue? Seems fairly easy, but tests still fail.

Graphite reactor would store information in Graphite.

Even if a reactor causes an error on each data event, the error handler is only triggered the first time.