Code Monkey home page Code Monkey logo

lockfile's Introduction

lockfile

A very polite lock file utility, which endeavors to not litter, and to wait patiently for others.

Usage

var lockFile = require('lockfile')

// opts is optional, and defaults to {}
lockFile.lock('some-file.lock', opts, function (er) {
  // if the er happens, then it failed to acquire a lock.
  // if there was not an error, then the file was created,
  // and won't be deleted until we unlock it.

  // do my stuff, free of interruptions
  // then, some time later, do:
  lockFile.unlock('some-file.lock', function (er) {
    // er means that an error happened, and is probably bad.
  })
})

Methods

Sync methods return the value/throw the error, others don't. Standard node fs stuff.

All known locks are removed when the process exits. Of course, it's possible for certain types of failures to cause this to fail, but a best effort is made to not be a litterbug.

lockFile.lock(path, [opts], cb)

Acquire a file lock on the specified path

lockFile.lockSync(path, [opts])

Acquire a file lock on the specified path

lockFile.unlock(path, cb)

Close and unlink the lockfile.

lockFile.unlockSync(path)

Close and unlink the lockfile.

lockFile.check(path, [opts], cb)

Check if the lockfile is locked and not stale.

Callback is called with cb(error, isLocked).

lockFile.checkSync(path, [opts])

Check if the lockfile is locked and not stale.

Returns boolean.

Options

opts.wait

A number of milliseconds to wait for locks to expire before giving up. Only used by lockFile.lock. Poll for opts.wait ms. If the lock is not cleared by the time the wait expires, then it returns with the original error.

opts.pollPeriod

When using opts.wait, this is the period in ms in which it polls to check if the lock has expired. Defaults to 100.

opts.stale

A number of milliseconds before locks are considered to have expired.

opts.retries

Used by lock and lockSync. Retry n number of times before giving up.

opts.retryWait

Used by lock. Wait n milliseconds before retrying.

lockfile's People

Contributors

felixsanz avatar iarna avatar isaacs avatar jskorepa avatar jsstorm avatar oresoftware avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lockfile's Issues

Bug in project => TypeError: cb is not a function

I just got this error

/home/olegzandr/Documents/WebstormProjects/poolio/node_modules/lockfile/lockfile.js:61
  fs.unlink(path, function (unlinkEr) { cb() })
                                        ^

TypeError: cb is not a function
    at /home/olegzandr/Documents/WebstormProjects/poolio/node_modules/lockfile/lockfile.js:61:41
    at FSReqWrap.oncomplete (fs.js:123:15)

Looking at the code, looks like it's a bug
https://github.com/npm/lockfile/blob/master/lockfile.js

if the user doesn't pass a callback to lockFile.unlock, we will get the error

seems reasonable that the user might not want to handle any unlock errors,
so the user would just call

lockFile.unlock(path);

instead of

lockFile.unlock(path, function(){});

especially since there will never be an error present in the callback given the code,

I will submit a PR

LOL not sure what just happened

I had this file on my project filesystem called server-pids.json

I want to (1) lock the file, (2) read from it (3) write to it, (4) then unlock the file, seems like standard stuff

the problem is, this code just deleted the file from the filesystem, is that expected?

  lockfile.lock(p, function (err) {                                // lock the file (?)
          fs.readFile(p, 'utf8', function (err, data) {
            if (err) {
              cb(err);
            }
            else {
              var pids;

              try {
                pids = JSON.parse(data);
                delete pids[pid];
              }
              catch (err) {
                console.log(err.stack || err);
                pids = {};
              }

              fs.writeFile(p, JSON.stringify(pids), function (err) {

                lockfile.unlock(p, function (err) {                               // unlock the file after writing (?)
                  if (err) {
                    console.log(err.stack || err);
                  }
                });
                if (err) {
                  console.log(' => fs.writeFile error => ', err.stack || err);
                  cb(err);
                }
                else {
                  cb(null);
                }
              });
            }

          });

        });

not sure what's up, but I laughed though

maybe if you delete files, it should be documented, or if it is then I didn't see it :)

[Question] About lockfile.check()

The async version says:

lockFile.lock('some-file.lock', opts, function (er) {
  // if the er happens, then it failed to acquire a lock.

So basically we are having a check in the callback.

But what about the sync version? We should do:

if (lockFile.checkSync(fileLocked)) {
  lockFile.unlockSync(fileLocked);
} else {
  console.log('er');
}

Or just:

if (!lockFile.unlockSync(fileLocked)) {
  console.log('er');
}

Stale option may not work correctly in windows

The stale option is checked using file creation time, which is sometimes incorrect in windows: If you delete a file and recreate the file with the same name within 15 seconds (default), the new file will have the same creation file as the old deleted one. this is said to be "working as intended", and called "Windows NT File System Tunneling", it affect all recent windows system from Windows XP to latest Windows 8 64bit.

See more details about this at:
http://support.microsoft.com/kb/172190/en-US
http://serverfault.com/questions/92757/incorrect-file-creation-date-in-windows-xp-vista

This will cause problems for lockfile when you lock the same file repeatly and are using the stale option.

sample code to always reproduce the problem:

process.env.NODE_DEBUG = "lockfile"

var lockFile = require('lockfile');
var fs = require("fs");

var lockFileName = "test.lock";

lockFile.lockSync(lockFileName, {stale: 10  * 1000});
console.log("lock done");
console.dir(fs.statSync(lockFileName));

setTimeout(function() {
    lockFile.unlockSync(lockFileName);
    console.log("unlock done");
    lockFile.lockSync(lockFileName, {stale: 10 * 1000});
    console.dir(fs.statSync(lockFileName));
    console.log("lock again");
    lockFile.lockSync(lockFileName, {stale: 10 * 1000});
    console.log("dup lock done");
}, 11 * 1000);

Expected result:
the last lockSync call will throw an error with EEXIST and the lock file stat is like:
atime: Fri Aug 23 2013 10:08:36 GMT-0400 (EDT)
mtime: Fri Aug 23 2013 10:08:36 GMT-0400 (EDT)
ctime: Fri Aug 23 2013 10:08:36 GMT-0400 (EDT)

atime: Fri Aug 23 2013 10:08:47 GMT-0400 (EDT)
mtime: Fri Aug 23 2013 10:08:47 GMT-0400 (EDT)
ctime: Fri Aug 23 2013 10:08:47 GMT-0400 (EDT)

Actual result:
show dup lock done and ends without exception, the lock file stat is like:
atime: Fri Aug 23 2013 10:09:45 GMT-0400 (EDT)
mtime: Fri Aug 23 2013 10:09:45 GMT-0400 (EDT)
ctime: Fri Aug 23 2013 10:09:45 GMT-0400 (EDT)

atime: Fri Aug 23 2013 10:09:56 GMT-0400 (EDT)
mtime: Fri Aug 23 2013 10:09:56 GMT-0400 (EDT)
ctime: Fri Aug 23 2013 10:09:45 GMT-0400 (EDT)

The work around i am using is to use the file modification time instead of creation time on windows. on other system, the mtime should also work but i only tested on linux.

Catch exit by Ctrl+C

If the script is exited by Ctrl+C, the lock is not removed.

If the script exits normally, it is caught by process.on('exit')

It should also listen to process.on('SIGINT') (linux at least).

Avoiding writeFile collisions?

Is it possible with this module?

I have a lot of asynchronous tasks that all write JSON to the same file in parallel. I will often get a corrupted file because two functions write to the file at the same time.

Is there anyway to avoid this using lockfile?

acquiring same lock but with different stale should fail

Example

  1. Lock (abc.lock) with stale = 10 seconds
  2. Wait 3 seconds
  3. Acquire the same Lock (abc.lock) with stale = 2 seconds

Expectation: Step 3 should fail
Actual: Step 3 passes (abc.lock is acquired)

Any solution for the above?

No changelog

Please add changelog. Especially for 1.0 release..

Library does not seem to fulfill contract

I was using this library for a bit, but it seems to cause problems when I make a lot of concurrent requests to get a lock on the same file. I get a lot of "file is already open" errors. Maybe the user (me) should handle those errors and do manual retries, but I was hoping that lockfile would do that tricky work for me.

So in order to get a more robust solution I started working on a mutex system with websockets.
I ran a performance test, and lockfile apparently is WAY faster than websockets. It's so fast that I am starting to doubt that it actually is working properly. lockfile can process 300 serially lock/unlock requests in less than 30ms, which seems way too fast to be possible. These are in series! Which means it can go through a lock/unlock cycle in 30ms / 300 locks which is 1/10 of a ms! Seems crazy. When Live-Mutex does the same, it takes ~500ms, so that's 500ms / 300 cycles which is about 2ms which seems much more normal.

Despite being fast, lockfile totally chokes immediately when these 300 serial requests are turned into concurrent/parallel requests. So I am very confused - lockfile seems impossibly fast, but also clearly chokes in totally normal circumstances, circumstances that require locking (aka non-serial access to a resource)!

I created the (speed) tests in this project:

https://github.com/ORESoftware/locking-lib-comparison

If you are curious, you can check it out, run npm install, and then run

git clone [email protected]:ORESoftware/locking-lib-comparison.git
cd project
npm install
./test.sh

the speed tests are pretty rudimentary, but basically I am bemused about why lockfile runs so fast when running through locks serially and why it barfs so quickly when try to gain access to a lock concurrently.

The whole point of locking is to manage concurrent access to a resource, which means the lock itself will experience concurrent requests!

Live-Mutex is slower, but apparently more robust?

[FEATURE] Refresh Locks

What / Why

Requesting that lock staleness always be checked by mtime for all platforms, and the ability to keep a lock refreshed by routinely touching it.

Example

lockfile.lock(file, {stale: 30000, refresh: 1000, retry: 1000*60}, (err) => {

});

While the lock is open routinely do fs.touch, then mtime (as used by windows) will recognize the lock is not stale. Unlock should then stop the active timer.

Support Promise API

With Promises being available in Node for a while now, it would be great if this library could return Promises.

Side note: has anyone tried this with util.promsify? Does that work as expected?

[FEATURE] Add more context in error messages

What / Why

It would be great if the reason the lock couldn't be acquired could be included in the error message. I thought there was something wrong with my implementation because I would intermittently see Error: EEXIST: file already exists, open 'xxx' in my logs. After extensive debugging, I realised it was working as designed, it was just timing out after the wait has expired.

I added the following workaround code:

  let wait = 60e3;

  let before = new Date();

  try {
    await lock(lockFilePath, { wait });
  } catch (err) {
    let elapsed = new Date() - before;

    if (err.code === 'EEXIST' && elapsed >= wait) {
      err = new Error(`\`${lock.name}\` failed to acquire lock after ${elapsed}ms. ${err.message}`);
    }

    throw err;
  }

It would be great if we could include all this context as to why it failed into the error message.

test staleness-sync-test failing on arm32

One of the tests in basic.js, staleness sync test, fails on amr32 hardware. This is seen on our setup with the following version:

Distribuiton: Debian Stretch
lockfile: 1.0.4-1

What I've noticed is that the Debian maintainer replaced the requirement for the touch module with the native calls to fs module. I doubt if something that simple would be the cause of the problem but I've anyways also asked the Debian Maintainer to help provide some information on why that change was made.

Currently, the test fails with the following error:

[  253s]     # Subtest: staleness test
[  253s]         ok 1 - expect falsey value
[  253s]         1..1
[  253s]     ok 7 - staleness test # time=12.937ms
[  253s]     
[  253s]     # Subtest: staleness sync test
[  253s]         ok 1 - expect falsey value
[  253s]         not ok 2 - EEXIST: file already exists, open 'stale-lock'
[  253s]           ---
[  253s]           stack: |
[  253s]             Object.exports.lockSync (lockfile.js:277:17)
[  253s]             Test.<anonymous> (test/basic.js:161:12)
[  253s]           at:
[  253s]             line: 439
[  253s]             column: 3
[  253s]             file: fs.js
[  253s]             function: Object.openSync
[  253s]           errno: -17
[  253s]           syscall: open
[  253s]           code: EEXIST
[  253s]           path: stale-lock
[  253s]           test: staleness sync test
[  253s]           ...
[  253s]         
[  253s]         1..2
[  253s]         # failed 1 of 2 tests
[  253s]     not ok 8 - staleness sync test # time=75.656ms
[  253s]     
[  253s]     # Subtest: retries
[  253s]         ok 1 - should be equal
[  253s]         1..1
[  253s]     ok 9 - retries # time=8.388ms
[  253s]     
[  253s]     # Subtest: retryWait
[  253s]         ok 1 - should be equal
[  253s]         1..1
[  253s]     ok 10 - retryWait # time=404.541ms

Question: how does retries work ?

Hi there,
Very simple question, I am using this great locking module with the following options:
{
wait: 1000,
stale: 20000,
retries: 1,
retryWait: 10
}
I was expecting the lock to fail after 2010ms but somehow it fails after far longer (around 5s). What did I misunderstood ?

Thanks

getting weird error - EEXIST: file already exists, open '/x/y/z.lock'

When I calllockfile.lock(z.lock)on a file that already exists, it is giving me this error in the callback

EEXIST: file already exists, open '/x/y/z.lock'

this is weird, because I thought the purpose of this lib was to wait for the file to be deleted and then reacquire the lock.

so confused about this one, any idea what it might be?

Works as intended across processes ?

I just realized, even though I have been using this lib for some time, that I am not sure if it works across (Node.js) processes. I am guessing it does, but I am not certain. Whatever is the case, it would be good to put this in the readme, I think. Something like:

  • Works across processes
    -or-
  • Only works in single processe

Race condition when using `stale` option

At the point that a lock file becomes stale, multiple processes are able to obtain the same lock.

Example code: https://gist.github.com/timmclean/8042936
This program locks test.lock once, then starts 16 concurrent attempts to lock test.lock (repeatedly attempting for 5s). Once the file becomes stale, multiple attempts to lock the same file will succeed, as shown in the output. Unhappiness ensues!

I believe this is due to the brief period of time between fs.stat, where the lockfile is determined to be stale, and unlock, where the lockfile is actually removed.

Consider the following sequence of events: Thread 1 (of course, my example doesn't use actual threads, but bear with me) detects that the lockfile is stale. Thread 2 detects that the lockfile is stale. Thread 2 deletes the lockfile. Thread 3 attempts to get a lock, and succeeds because of Thread 2's deletion. Thread 2 tries to get the lock and fails. Thread 1 deletes the lockfile because it think that it's still stale. Thread 1 tries to get the lock and succeeds. As a result, both Thread 1 and Thread 3 now have the same lock.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.