Strange benchmark result about asyncawait HOT 9 CLOSED

callbacks are about twice as fast on my setup.

What about if you set var USE_MOCK_FS = true; (line 45)?

That should rule out any fs wierdness. By default it's couting files in the same folder as benchmark.js, which is only three files. There may be cache effects, etc etc. You could also try counting files in a bigger dir - set dirToCheck on line 201.

from asyncawait.

Some more results
USE_MOCK_FS = true
19312.47585940518 samples/sec callback
20559.21052631579 samples/sec sync
It's ok.

1000 files (USE_MOCK_FS = false)
50.994390617032124 samples/sec callback
76.26019980172349 samples/sec sync
still strange

NOTE:using ssd

from asyncawait.

Yes strange. I also have an SSD.

You could write a standalone .js file that does a similar test using fs sync/async functions if you are really keen to investigate!

from asyncawait.

I rewrited tests
https://gist.github.com/vird/db50ff47f0066903f34e

1439.2630972941854 sync
2337.5409069658717 callback
That's ok.

from asyncawait.

Interesting. Note that your code is doing something different. The fs.stat calls are used to filter out directories and only count files.

from asyncawait.

Looking at my version of countFiles-callbacks.js, I think it could be made significantly faster if it didn't create a new closure on every call (ie function getStatsInParallel could be moved out and take paths as an argument).

from asyncawait.

Try this countFiles-callbacks.js. Does it make any difference?

var fs = require('fs');
var path = require('path');
var _ = require('lodash');


/** Returns the number of files in the given directory. */
var countFiles = function (dir, callback) {

    // Get all directory entries.
    fs.readdir(dir, function (err, files) {
        if (err) { callback(err); return; }
        var paths = _.map(files, function (file) { return path.join(dir, file); });
        getStatsInParallel(paths, function (err, stats) {

            // Count the files.
            if (err) { callback(err); return; }
            var result = _.filter(stats, function (stat) { return stat.isFile(); }).length;
            callback(null, result);
        });
    });
}


function getStatsInParallel(paths, callback) {
    var stats = [];
    var remaining = paths.length;
    if (remaining === 0) {

        // Directory is empty.
        callback(null, stats);
        return;
    } else {

        // Get all file stats in parallel.
        paths.forEach(function (path, i) {
            fs.stat(path, function (err, stat) {
                if (remaining === 0) return;
                if (err) {
                    remaining = 0;
                    callback(err);
                } else {
                    stats[i] = stat;
                    --remaining;
                    if (remaining === 0) callback(null, stats);
                }
            });
        });
    }
}



module.exports = countFiles;

from asyncawait.

previous check
50.994390617032124
Old (rechecked)
52.36424569304079
So test has +-4% tolerance

New implementation
52.53480430785395 samples/sec

with mock-fs
21682.56721595837 samples/sec
new callback

vs previous
19312.47585940518 samples/sec callback
4%*20000 == 800
2300 > 800 so, improvement detected

from asyncawait.

It may not be that surprising if sync code beats callback code in some scenarios. The callback code creates many heap objects (eg closure instances) as it runs, so will have higher CG pressure etc. If the task itself (in this case readdir and stat) is very fast, which it may be with caching, etc, then the sync code has some advantages that may give it the edge overall. Bear in mind that fs async ops are not as concurrent as they look in the code, I think by default only about 4 fs operations can be truly handled in parallel by libuv.

I suppose if you really wanted a clear answer, you could do some profiling with CPU sampling to see where the most time is being spent in each case.

from asyncawait.