learnboost / cluster Goto Github PK

Hi!

Please, consider adding a duplex way for workers to broadcast simple JSON messages. It may be achieved via a shared unix socket, like it's done at https://github.com/dvv/simple/blob/master/src/server.coffee#L194 for master and at https://github.com/dvv/simple/blob/master/src/server.coffee#L77 for workers.

I just discovered cluster and want to switch to it and to cease maintaining own solution, but I need intercommunication so that workers can report to each other changes in their state. Having redis just for pubsub seems overhead. Another pubsub solutions are either immature or too complex to employ.

TIA,
--Vladimir++

currently the new master kicks in on restart, clobbers the old one in terms of the REPL, so either we change the path for the old one so you can still check the status, or via IPC grab stats from the previous master

I haven't started using cluster yet, but I saw this in worker.js:

process.on('uncaughtException', function(err){
    console.error(err.stack || err.message);
    self.destroy();
});

This should be optional because (if I'm not wrong... oh, asynchronous stuff), self.destroy() might be called before exceptional-node's sending call.

.

lots of procs :)

Add optional but transparent support for duplicating the server on 443 with the given options

Seeing some unexpected behavior when attempting to restart master / workers programatically. I've summarized the problem here: https://gist.github.com/e4ef8fcbb5be2f468967

Is this an issue or am I doing something wrong?

Hi, I want to use http://learnboost.github.com/cluster/ on my server, but with the demo code on github, I get this error message:

events.js:23
throw arguments[1]; // Unhandled 'error' event
                   ^
Error: ENOTSUP, Operation not supported
at Server._doListen (net.js:1062:5)
at net.js:1010:16

NodeJS version is v0.4.2.

app.js:

var http = require('http');
module.exports = http.createServer(function(req, res){
  console.log('%s %s', req.method, req.url);
  var body = 'Hello World';
  res.writeHead(200, { 'Content-Length': body.length });
  res.end(body);
});

server.js:

var cluster = require('cluster')
, app = require('./app');
cluster(app)
  .use(cluster.logger('logs'))
  .use(cluster.stats())
  .use(cluster.pidfiles('pids'))
  .use(cluster.cli())
  .use(cluster.repl(8888))
.listen(3000);

Can anybody help?

master does not need to require everything that the user's server does, does not need to connect to databases etc, so this can cause master to hang when restarting etc. We have a few options... including supplying a path to cluster() to load the app but ONLY in the workers.

regardless we need docs for this

The simplest server test.js:

var cluster, http, server;
cluster = require('cluster');
http = require('http');
server = http.createServer(function(req, res) {
  var body;
  console.log('%s %s', req.method, req.url);
  body = 'Hello World';
  res.writeHead(200, {
    'Content-Length': body.length
  });
  return res.end(body);
});
cluster(server).set('workers',4).use(cluster.debug()).use(cluster.reload(['test.js'])).use(cluster.stats()).use(cluster.repl(30000)).listen(3000);

Below is typical debug output when starting workers. What can be the reason?

info �[90m- master started�[0m
info �[90m- worker 0 spawned�[0m
info �[90m- worker 1 spawned�[0m
info �[90m- worker 2 spawned�[0m
info �[90m- worker 3 spawned�[0m
info �[90m- worker 0 connected�[0m
info �[90m- worker 3 connected�[0m
Error: EINVAL, Invalid argument
at IOWatcher.callback (net.js:878:24)
�[31merror�[0m �[90m- worker 0 uncaught exception EINVAL, Invalid argument�[0m
Error: EINVAL, Invalid argument
at IOWatcher.callback (net.js:878:24)
�[31merror�[0m �[90m- worker 3 uncaught exception EINVAL, Invalid argument�[0m
�[33mwarning�[0m �[90m- worker 0 died�[0m
info �[90m- worker 0 spawned�[0m
�[33mwarning�[0m �[90m- worker 3 died�[0m
info �[90m- worker 3 spawned�[0m
info �[90m- worker 1 connected�[0m
info �[90m- worker 2 connected�[0m
info �[90m- listening for connections�[0m
Error: EINVAL, Invalid argument
at IOWatcher.callback (net.js:878:24)
�[31merror�[0m �[90m- worker 2 uncaught exception EINVAL, Invalid argument�[0m
Error: EINVAL, Invalid argument
at IOWatcher.callback (net.js:878:24)
�[31merror�[0m �[90m- worker 1 uncaught exception EINVAL, Invalid argument�[0m
�[33mwarning�[0m �[90m- worker 2 died�[0m
info �[90m- worker 2 spawned�[0m
�[33mwarning�[0m �[90m- worker 1 died�[0m
info �[90m- worker 1 spawned�[0m
info �[90m- worker 3 connected�[0m
info �[90m- worker 0 connected�[0m
info �[90m- worker 1 connected�[0m
info �[90m- worker 2 connected�[0m
info �[90m- shutting down�[0m
�[33mwarning�[0m �[90m- kill(SIGKILL)�[0m
info �[90m- shutdown complete�[0m
�[33mwarning�[0m �[90m- worker 3 died�[0m
�[90mdebug - exit�[0m

I created the following CLI command to attach the node debugger but it doesn't seem to be working and I'm not sure why.

cluster.cli.define('-d, --debug, debug', function(master) {
  master.kill('SIGUSR1');
  process.exit();
}, 'Attach the node debugger to workers');

I was under the impression that signalling the worker processes with SIGUSR1 should attach the debugger. The bottom of this page explains it: http://nodejs.org/docs/v0.4.1/api/debugger.html

What ends up happening is that all workers are restarted. I didn't think that master.kill('SIGUSR1') would do that. Is there another way to attach the debugger to workers?

What I would really love is for it also to work after workers reload from the 'reload' plugin. Possible?

I'm finding that when my app is restarted, either via the cli or automatically by reload, require.paths is reduced to:
[ '/usr/local/lib/node' ]
down from:
[ '/Users/sam/.npm',
'/Users/sam/.node_modules',
'/Users/sam/.node_libraries',
'/usr/local/lib/node' ]
causing "Cannot find module" errors.

I'm guessing this has something to do with process.ENV being replaced with:
{ CLUSTER_REPLACEMENT_MASTER: '1',
CLUSTER_PARENT_PID: '590',
__CF_USER_TEXT_ENCODING: '0x1F5:0:0' }

currently the events fire, but they do not allow the shutdown to halt until cleanup is complete, removing files, sockets etc

title() and row()

for example when a user specifies a dir for log output, it may not exist, implement mkdir -p

waiting on this node patch: https://github.com/ry/node/pull/674/files

it would be nice to list the workers and their current state, aka active, shutting down etc

as per guillermo's request :) this is a good idea though, telnet seems to suck on our server

doesn't matter what I trying to do, I get always this

events.js:23
        throw arguments[1]; // Unhandled 'error' event
                       ^
Error: EINVAL, Invalid argument
    at Server._doListen (net.js:1062:5)
    at net.js:1010:16

now using node v0.4.0, on macos.

I have tried your examples and to run tests, nothing worked, always with the same error.

My Express application starts with no errors and spawns the workers but the site is not accessible when started under Cygwin.

On MacOS it works perfectly.

logger() exposes the streams, so we could add logrotate()

allowing cluster to require the exported server only when in a worker to prevent these issues with db connections being open in master etc

Feasible to make spawn honor negative numbers to decrease the number of workers?

--Vladimir++

I am using latest Express with cluster, my node version is v0.4
var cluster = require('cluster');
var express = require('express');
var app = module.exports = express.createServer();

    app.get('/', function(req, res){

      console.log('New Request'); // this does not output

      res.render('index', { title: 'Express' });
    });

    cluster(app)
        .use(cluster.logger('logs'))
        .use(cluster.stats())
        .use(cluster.pidfiles('pids'))
        .use(cluster.cli())
        .use(cluster.repl(8888))
        .use(cluster.reload(['app.js']))
        .use(cluster.debug())
        .listen(3000);

It doestn't output the message using console.log() in the console or master.log or workers.access.log.
I am not sure it is a bug or by design?
Thanks

Hi,

I tried to run the simple example in Readme.md on my computer (Ubuntu 10.10, 64bits) with node 0.4.0, but I get the following error (appears multiple times) :

(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace:
at [object Object]. (events.js:101:17)
at Socket.pipe (stream.js:67:8)
at Master. (/home/tanguy/local/lib/node/.npm/cluster/0.0.3/package/lib/plugins/logger.js:136:21)
at Master.emit (events.js:59:20)
at Master.spawnWorker (/home/tanguy/local/lib/node/.npm/cluster/0.0.3/package/lib/master.js:310:8)
at Master.workerKilled (/home/tanguy/local/lib/node/.npm/cluster/0.0.3/package/lib/master.js:461:12)
at Master.invoke (/home/tanguy/local/lib/node/.npm/cluster/0.0.3/package/lib/mixins/receiver.js:51:18)
at Master.frame (/home/tanguy/local/lib/node/.npm/cluster/0.0.3/package/lib/mixins/receiver.js:32:14)
at Socket. (native)
at Socket.emit (events.js:42:17)

Here is an extract of master.log

[Wed, 16 Feb 2011 23:32:10 GMT] INFO master started
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 0
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 1
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 2
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 3
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 4
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 5
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 6
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 7
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 3 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 3
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 1 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 1
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 0 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 0
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 5 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 5
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 2 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 2
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 6 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 6
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 4 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 4
[Wed, 16 Feb 2011 23:32:10 GMT] ERROR worker 7 died
[Wed, 16 Feb 2011 23:32:10 GMT] INFO spawned worker 7
[Wed, 16 Feb 2011 23:38:26 GMT] ERROR worker 7 died
[Wed, 16 Feb 2011 23:38:26 GMT] INFO spawned worker 7
[Wed, 16 Feb 2011 23:38:26 GMT] ERROR worker 6 died

Not sure if this is cluster-specific though...

Thanks in advance!

http://www.geckoboard.com/

we would loose some cool functionality with this ... BLAH

node version v0.3.7, macos

node.js:116
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Cannot find module 'log'
at Function._resolveFilename (module.js:204:11)
at Function._load (module.js:162:25)
at require (module.js:231:19)
at Object. (/usr/local/lib/node/.npm/cluster/0.0.2/package/lib/plugins/logger.js:13:11)
at Module._compile (module.js:287:26)
at Object..js (module.js:293:10)
at Module.load (module.js:219:31)
at Function._load (module.js:186:10)
at require (module.js:231:19)
at Function.logger (/usr/local/lib/node/.npm/cluster/0.0.2/package/lib/cluster.js:46:12)

Currently they work only in the master process, so you cannot check server connections etc. it would be expensive to report such things back, but plugins should still be able to function at the worker level, enabling active connection stats etc

os is currently required for some stats

could be third-party

I'm using the simple example app.js you give; server.js is just
var cluster = require('cluster')
cluster('./app').listen(8001)

I get the following error:

$ node server.js 

node.js:116
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
TypeError: Cannot read property 'length' of undefined
    at Master.start (/usr/local/lib/node/.npm/cluster/0.3.2/package/lib/master.js:393:18)
    at /usr/local/lib/node/.npm/cluster/0.3.2/package/lib/master.js:253:16
    at Master.createSocket (/usr/local/lib/node/.npm/cluster/0.3.2/package/lib/master.js:366:5)
    at Master.listen (/usr/local/lib/node/.npm/cluster/0.3.2/package/lib/master.js:250:14)
    at Object.<anonymous> (/usr/home/elijah/cluster/server.js:3:18)
    at Module._compile (module.js:373:26)
    at Object..js (module.js:379:10)
    at Module.load (module.js:305:31)
    at Function._load (module.js:271:10)
    at Array.<anonymous> (module.js:392:10)

It looks like os.cpus() is undefined in
if (!this.has('workers')) {
this.set('workers', os
? os.cpus().length
: 1);
}

could be third-party

adds a LOT of stat watchers

currently cluster (by default) writes to some files in the script's directory, causing a cyclic restart. Template engines + css etc already can be modified on the fly so we should only really watch for *.js files

example:

cluster(server)
  .set('workers', 4)
  .in('development').use(cluster.logger('logs', 'debug'))
  .in('development').use(cluster.debug())
  .in('production').use(cluster.pidfiles())
  .listen(3000);

 $ node app.js status

etc

tough call here, hard to rely on node for things like this, maybe just best to leave monit in the picture but for those who have small sites this would be a nice feature, or if node can actually become stable enough to rely on the workers :)

in cli plugin, "-s" is used for both status and shutdown commands

currently if you SIGKILL a worker, master is not reported, and ha no way of knowing it lost a worker. Again we need the null signal support in the upcoming node 0.4.1 to allow the worker count to be maintained, until then, we have to rely on the workers contacting master when they die

When starting a cluster in the background (nohup & ...) and then using the cli plugin to restart the cluster, old master and workers processes aren not killed after the new main process has taken over. This can be reproduced with examples/cli.js as follows:

$ nohup node cli.js &
$ node test.js restart
debug - exit
$ node test.js restart
debug - exit
[1]+ Done nohup node test.js
$ node test.js restart
User defined signal 2
$ node test.js restart
User defined signal 2

In the meantime the amount of processes has already doubled/tripled... all the time no (persistent/browser) connections have been made - the above was just executed quickly one command after another.

currently all n workers must be connected

.

as a plugin, but we need 0.4.1 for the null signal patch