Playing with some readable streams and async iterators examples and creating this for tomorrow when I have probably forgotten everything already.

Not great for buffering, there's no flow control. Data is read as quickly as possible.

function readFile(filename) {
  const stream = createReadStream(filename);

  stream.on('data', (chunk) => {
    console.log(chunk);
  });
}

By using the readable event, the consumer has the freedom of deciding whether to read or not from the stream when the event is triggered.

function readFilePolling(filename) {
  const stream = createReadStream(filename);

  stream.on('readable', () => {
    let data;
    while ((data = stream.read()) !== null) {
      console.log(data);
    }
  });
}

A very elegant option to using the readable event is to use async iterators. The consumer still has control on how data is consumed and additionally it brings more readability.

async function readFileAsyncIterator(filename) {
  let chunk;
  for await (chunk of createReadStream(filename)) {
    console.log(chunk);
  }
}

Now, instead of reading information from files, let's read it from generators. This time let's just focus on the async iterators strategy.

Generator example:

function* generate() {
  for (let i = 0; i <= 30; i++) {
    yield i;
  }
}

Read with async iterator:

async function readStreamFromAsyncIterator() {
  const stream = Readable.from(generate());
  let chunk;
  for await (chunk of stream) {
    console.log(chunk);
  }
}

Note that Readable.from gets an iterable as parameter. We could pass an array for example.

Also note that it's not convenient to use this approach just to iterate over the values generated by a generator, as generate() is iterable so doing something like this would be more appropriate:

for (chunk of generate()) {
  console.log(chunk);
}

Combining async generators and async iterators is a quite clean option when the generator needs to be async (not the case of the example, though).

Async generator example:

async function* asyncGenerate() {
  for (let i = 0; i <= 30; i++) {
    yield i;
  }
}

Read with async iterator:

async function readAsyncGenerator() {
  for await (const chunk of asyncGenerate()) {
    console.log(chunk);
  }
}

• Stream Into the Future (NodeJS Streams)
• Node streams documentation