ricea / encoding-streams Goto Github PK

With this change the Encoding Standard will have two concepts of "stream": https://encoding.spec.whatwg.org/#concept-stream and a TransformStream.

I see three ways out of this:

• Rename the Encoding Standard's concept to something like "token sequence" and update all existing uses.
• Try to consolidate everything into the TransformStream model internally.
• Create some underlying concept that both can share, such that TransformStream is one implementation of the concept and TextEncoder/TextDecoder are another.

I think I lean toward the "token sequence" approach. The way it is treated is pretty different from streams in general, e.g. allowing both prepending and appending, and using EOS instead of { value, done }, and using abstract single bytes and single code points instead of JS Uint8Arrays and strings (which can themselves represent many characters each). This fits with my preference expressed in whatwg/encoding#72 (comment) to have a clean break between the "token sequence APIs" (TextEncoder/TextDecoder) and the TransformStream APIs (TextEncoder.stream/TextDecoder.stream).

@annevk should probably weigh in on this.

I'd always envisioned that the chunks would be "roughly" the same size on both the readable and writable sides, with perhaps some trailing bytes left over when decoding. This spec, however, seems to say that the readable side's chunks will always be created by processing a single token, and enqueuing each.

Is this intended? Are we worried about this behavior? Is it better, or worse, for implementers?

I was trying to use this in Chrome with JavaScript and get "not defined" Is there an include I need somewhere? I am trying this in the Glitch envitonment.

For associated "internal slots" (decoder, stream, ignore BOM flag, BOM seen flag, and error mode), their defining instance should be a <dfn>, and all subsequent uses of or references to them should be <a>s. Currently both are <b>. (The existing spec seems bad at this.)

The "decode and enqueue a chunk" and "flush and enque" algorithms should take a transformer object as an argument transformer, and then they can refer to the "internal slots" as e.g. transformer's decoder.

We'll probably want special phrases for doing things like constructing a TransformStream and calling the controller methods, similar to https://fetch.spec.whatwg.org/#readablestream. However I still think I should move those into the stream spec: whatwg/streams#372 (comment). For now the style you used is fine.

Returns the result of running encoding’s decoder on the chunks written to writable. Chunks read from readable will always be strings.

I'd go with something that mentions the type here, e.g.

Returns a readable stream whose chunks are strings resulting from running encoding's decoder on the chunks written to writable.

I think a paragraph break before the "Typically this will be used" paragraphs might be nice.

Mentioning USVString in developer-facing documentation is probably not ideal. Just "string" is fine.

It's not entirely clear that the "transform" concept holds a JavaScript TransformStream object. Making that clearer would help.

Then,

The readable attribute’s getter must return the readable attribute of transform.

and its friend should probably be revised. I'm not sure whether to just do a small revision, like s/attribute/property, or something more extensive, like

The readable attribute's getter must return the value of the readable property of the transform object, i.e. the value returned by Get(transform, "readable"). NOTE: since the transform object is unobservable, this value can be cached.

Links from each to the other would be nice :)

All interface and method names, e.g. ReadableStream and writable, should be monospace. In Bikeshed, this is done by using {{ReadableStream}} and {{TextDecoder/writable}} and similar.

This may be best solved in the process of doing a PR since otherwise there can be cross-linking errors (especially for TextDecoder/writable and its friend).

In particular it should probably use https://streams.spec.whatwg.org/#is-readable-stream-locked and https://streams.spec.whatwg.org/#is-writable-stream-locked. Plus we have the imprecision in how you refer to the readable and writable properties as noted in #5.

Maybe we need to define centrally the concepts of a TextDecoder's readable and a TextDecoder's writable.

In the DOM intro, TextDecoder should be monospace and link to the TextDecoder constructor.

RangeError and TypeError should link throughout.

TransformStream and ReadableStream should be monospace and link throughout. (Although TransformStream doesn't have anywhere to link to yet, so I guess disregard that part.)

Uint8Array and ArrayBuffer should link.

In "UTF-8's encoder", both "UTF-8" and "encoder" should link.