stream-utils / stream-to-array Goto Github PK

As part of a process of alignment with the tc39 iterator helpers proposal - just letting you know.

The main difference is that this doesn't listen to data (which we believe is better) and instead uses the async iterator protocol (which in turn uses readable).

nodejs/node#41553

Feedback welcome

Couldn't this be simplified to something like:

var toArray = function() {
  return someStreamReducer(function(arr, item){
    arr.push(item);
    return arr;
  }, []);
};

stream
.pipe(toArray())
.on('data', function(array){
});

Just confused with all the additional work in this lib unless I'm missing something.

This module depends on both native-or-bluebird and bluebird, but only the former is actually required in code.

terrible

The stream-to npm module points to a stream-to git repo that redirects here.

Ideally the README in that npm module would be updated to note it is deprecated (maybe remove the repo reference as well), and also include information about what modules replace its functionality. stream-to-array seems to be the replacement for streamTo.array and raw-body appears to be the replacement for streamTo.string and streamTo.buffer.

https://github.com/kevinbeaty/any-promise

this implementation supports more promise implementations and allows the end-user to configure the promise implementation.

Thanks for this library. I've found the following issue after using stream-to-promise that depends on this library.

Consider the following code that has a paused readable stream containing data at the time of calling toArray():

const MiniPass = require('minipass');
const toArray = require('stream-to-array');

const mp = new MiniPass();
mp.end('foo');
toArray(mp).then(parts => console.log(parts));

The promise never resolves.

On digging, it appears that, within stream-to-array, event listeners are registered in the following order:

stream.on('data', onData)
stream.on('end', onEnd)

However, if I change the order of listener registration ('end' before 'data') then the promise will resolve.

From the docs for Streams at https://nodejs.org/api/stream.html:

The 'data' event is emitted whenever the stream is relinquishing ownership of a chunk of data to a consumer. This may occur whenever the stream is switched in flowing mode by calling readable.pipe(), readable.resume(), or by attaching a listener callback to the 'data' event.

The 'end' event is emitted when there is no more data to be consumed from the stream. The 'end' event will not be emitted unless the data is completely consumed. This can be accomplished by switching the stream into flowing mode, or by calling stream.read() repeatedly until all data has been consumed.

I think it follows that, if there is data available on a non-flowing readable stream, then registering the data handler will cause it to start flowing and potentially result in the 'end' event firing immediately.