arbitrary connection support about readr HOT 5 CLOSED

Sounds like this is going to be a lot of work, and we'll need to implement quite different strategies for mmap files and connections. I think this means that connections should be at least a 0.2 feature.

from readr.

Sure. I'll focus on mmap for now but keeping in mind extensibility. mmap is somewhat orthogonal to the processing. mmap is just a way to get a char*. The big win of mmap is that we get at once a pointer for all (or at least a big chunk of the data), for other connections I'll have to either do things char by char, or line by line using Rconn_fgetc or something.

But anyway, if we move this to a 0.2 version, then perhaps this gives me time to worry about connections upstream in Rcpp implementations.

from readr.

Sounds good to me. I wonder if there's any support for mmapped gz/bz2 files. That's going to be a pretty common use case in my experience.

from readr.

I don't think so, which is a reason why I'd want to be able to consume a connection through R internal api.

from readr.

Apart from correct handling of the connection push back #19 I've leveraged what I need from the connection api.

from readr.