Comments (5)
There are two points of view here:
- the consumer needed that many bytes, and returning less than what is requested is an error, so it should fail
- the consumer can decide whether there is enough data or not. Sometimes you request more data but don't know how much you need (ex: byte separators).
I would be more inclined to the second option, since it helps for more parsers. In both cases, though, it should not fail allocating the slice. I even think it should always return the largest slice possible (all of the accumulated bytes) and let consume()
decide where to stop.
What do you think?
from nom.
I agree: The second option is clearly superior in terms of flexibility. The issue that I see with that is that existing code might do something like if input.len() < bytes_needed { ConsumerState::Await(0,bytes_needed) }
(or the equivalent thing by pulling the size from an IResult::Incomplete()
), and if you repeatedly return the maximum available input if you don't get enough new data to fulfill the consumer's request such code would thrown into an endless loop (as opposed to panicking, as it does now). So you would have to track whether you've already asked for that input and not gotten it, which is clumsy – especially since run()
already knows about that.
In short, run()
would need a way to inform consume()
about whether the last Await/Seek operation could be fulfilled. I can't think of a non-API breaking way to do that, but what might work would be to wrap the input in an enum telling consume()
about the way the new input came to pass. This could be something like the following (off the top of my head, probably nonsense and/or needs some 'a
sprinkled around). That would also be a nice opportunity to pass ProducerErrors down into consume()
:
enum InputChunk {
/// The entire requested data was produced
Complete(&[u8]),
/// None or only a part of the requested data was produced
Incomplete(&[u8]),
/// Encountered an error while producing data, some data
/// might still have been produced
Error(&[u8], ProducerError)
}
Alternatively, you could have callbacks about incomplete input, but I really can't come up with an idea that feels optimal.
from nom.
FYI I fixed the crash in 9f5128c, but I will not modify much the current design of producers and consumers, since I plan to change them completely for the 1.0 version. They're rather slow, and not really flexible, and I want to change that. The good news is that your suggestions on a new design are very welcome :)
from nom.
Please indicate your suggestions in #80 :)
from nom.
The stream feature will be removed in the next nom version (4.0), since commit 564a934. If someone wants to extract the code and maintain it, feel free to do it, it's in src/stream.rs
. Otherwise, I would recommend that you check out the (currently in nightly) generators feature, which is much nicer to use. Here's an example of using nom with generators.
from nom.
Related Issues (20)
- unsigned integer underflow HOT 3
- using `.clone()` on a double reference HOT 1
- Nom's error is referencing data owned by the calling function
- Any other approach to handle left recursion
- How to parse until a range of tags HOT 3
- Mention difference between is_not and take_until in their respective docs.
- I was going to complain about an issue in the function take_while_m_n, but [update : and I will do] ...
- Add u48 number parser HOT 1
- I want to implement `separated_tuple` for 8.0 HOT 7
- `tag_no_case` panicks while matching certain unicode capitalized characters.
- Ergonomics of `alt()` taking tuple versus slice HOT 3
- A new release for 8.0?
- FromStr-esque parsing trait HOT 1
- Float parsing with trailing `e` broken
- A fix has to be implemented by those dependencies HOT 7
- FlatMap does not properly implement Parser HOT 2
- Feature Request: Add branching parser combinator for `exclude()` and `depend()` HOT 1
- Document (or fix?) the fact that escaped() succeeds on empty inputs HOT 7
- Stack overflow before parsing HOT 1
- fold's param init func should be FnOnce instead of FnMut HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nom.