71104 / rwlock Goto Github PK

Looks like a memory leak? It's only happened once so I'm not 100% sure it's rwlock that's actually leaking, but it would make sense that it is - I'm creating lots of one-time use locks with unique identifiers

<--- Last few GCs --->

127398132 ms: Scavenge 1534.7 (1587.0) -> 1534.7 (1587.0) MB, 0.2 / 0 ms (+ 2.5 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
127401398 ms: Mark-sweep 1534.7 (1587.0) -> 1352.4 (1587.0) MB, 3266.6 / 0 ms (+ 3.4 ms in 2 steps since start of marking, biggest step 2.5 ms) [last resort gc].
127404780 ms: Mark-sweep 1352.4 (1587.0) -> 1527.7 (1582.0) MB, 3381.7 / 0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x9088c1b4629
1: writeLock(aka c) [/home/kieran/btc/broadcaster/node_modules/rwlock/lib/lock.js:~5] [pc=0x11e6eff43898](this=0x197eb8f2c61 ,b=0x111e36fb7e21 <String[10]: 25-2141721>,c=0x111e36fb7dd9 <JS Function %28SharedFunctionInfo 0x13c646a98ed9%29>,f=0x9088c1041b9 <undefined)
2: arguments adaptor frame: 2->3
3: /* anonymous */ [/home/kieran/btc/broadcaster/index.js:285] [pc=0x11e6ee63...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted (core dumped)

As title, if a node application is clustered to maximise performance, will this module still work in this case or it is just for single process node application?

This looks to only work for a single process? I don't see any i/o routines in the library.
Since node.js is single-threaded, why do we need a lock for a single process?

Alberto

I have a node application that has a concurrency problem that I thought I might be able to solve with your package but I'm not sure it is possible. In handling requests, my application creates various cache files which are used for speeding up subsequent requests. The problem comes about when I get 2 (or more) requests in a short time period which attempt to create the same cache data. Things work as expected if the time between requests is great enough such that the initial request has the time to create the cache.

I thought about using a lock file to control the cache creation but I don't want subsequent requests to fail if the lock file cannot be obtained. I would rather that the first request be able to create a lock such that subsequent requests would be wait until the initial request is done handling the cache creation.

Here is a sample function which passes in a value which can be obtained from parsing a url. I've ignored all errors in my sample code for simplicity. If the markerFile does not exist it means that the cache doesn't exist. If the markerFile exists, someDataFile contains valid data.

function doProcess(someValue, callback)
{
var markerFile = 'c:\temp\marker';
var someDataFile = 'c:\temp\data';

  fs.exists(markerFile, function (exists) {
     if (exists) {
        fs.readFile(someDataFile, function(err, data) {
           // ignore error
           console.log('data:  ' + data);
           performCalculation(someValue, data, function(err, result) {
              // ignore error
              console.log('result:  ' + result);
              if (callback) {
                 callback(result);
                 }  
              });
           });
        }  
     else {
        createSomeData(function(err, data) {
           console.log('data:  ' + data);
           fs.writeFile(someDataFile, data, function(err) {
              // ignore error
              performCalculation(someValue, data, function(err, result) {
                 // ignore error
                 console.log('Result:  ' + result);
                 // all done create test file
                 fs.writeFileSync(markerFile, '');
                 if (callback) {
                    callback(result);
                    }
                 });
              });
           });
        }
     });
  }

If I have a browser open and I have a url which gets parsed and calls the above function, things work as expected. However, if I have 2 browsers open with urls which get parsed which call the above function and my markerFile doesn't exist, it is possible that both calls into the function with attempt to create the data file...especially if it takes some time to generate someDataFile.

So can your library be used to wrap a locking mechanism around things such that we can guarantee that someDataFile will only be created once and that other calls will wait until the markerFile is created?

thanks

bill

You have called lock.queue[0]() in the release already.