azilber / always-tail2 Goto Github PK

I've been using always-tail2 to process log files in real time, and for hours now I've been running into this issue in which I get the following error when running the logger at normal speed (rather than slowed down):

buffer.js:495
    throw new Error('"toString()" failed');
    ^

Error: "toString()" failed
    at Buffer.toString (buffer.js:495:11)
    at project-path/node_modules/always-tail2/index.js:80:33
    at FSReqWrap.wrapper [as oncomplete] (fs.js:682:17)

To the best I can tell, this occurs because of a string size limit in V8, which is run into whenever the output stream hasn't been flushed in long enough for incoming data to accumulate past the limit. This error happens right here in always-tail2:

var buffer = new Buffer(size);

debug('reading:', block.fd, size, start);
fs.read(block.fd, buffer, 0, size, start, function(err, bytesRead, buff) {
    var chunk, parts, _i, _len, _results;

    if (err) {
        return self.emit('error', err);
    }

    if (bytesRead == 0) {
        return next()
    }

    self.bookmarks[block.fd] += bytesRead;
    buff = buff.toString('utf-8');      // <- This line is where the actual error occurs
    self.buffer += buff;
    parts = self.buffer.split(self.separator);
    self.buffer = parts.pop();
    _results = [];

    for (_i = 0, _len = parts.length; _i < _len; _i++) {
        chunk = parts[_i];
        _results.push(self.emit('line', chunk));
    }

    next();
});

If the update size could be checked ahead of time and the incoming data split into multiple buffers if it exceeds the string size limit, and then splits and output stream writes processed sequentially on each of the buffers, this error would be impossible so long as no single line of the tailed file passes the string size limit. The only additional complexity this would add is guaranteeing that lines are pushed in the correct order.

The debug dependency has vulnerabilities. These have been fixed in v2.6.9. Can you update the dependency please?

Many thanks.