Incremental input about combine HOT 5 CLOSED

I'm interested in getting this working as I'd like to use parser combinators to parse XML from a serial stream...

One idea I had was to use an intermediate struct in place of the input - something that buffers input items as they're read.

struct BufferedStream<Input, Stream> {
    buffer: Vec<Input>,
    stream: Stream
}

Then parsers could read from the buffer directly if it were non-empty, or else pull a new item from the stream. It prevents the need to "reverse" changes to the stream if a pulled token ends up being unparseable, as it can simply be thrown onto the buffer (no stream cloning required). I will admit I haven't given this a whole lot of thought though... I'll play around with it a bit more.

from combine.

Or we could steal a bunch of ideas from nom, which seems to have streaming but isn't generic.

from combine.

I might be wrong but I don't think your BufferedStream handles parsers which can consume multiple tokens before failing. If you put the token in the buffer if it fails there might still have been tokens which has already been accepted which can't be recovered at the point that parsing is resumed.

I am not sure that the way nom handles streaming works well with recursive descent parsers (which is what parser combinators are, at least in this library). If a parser needs more input in nom I think it basically throws away everything it has parsed up to that point, requests more and then continues once it gets it (I might be wrong though) if you are deep within a parser when that point lots of work would be repeated.

from combine.

@michaelsproul I have a sketch for how this might be implemented in #37. I think it is rather close to what you had in mind, any comments?

from combine.

Closed by 2.0.0

from combine.