(csv) Add a `CsvParser.Feature.SKIP_EMPTY_LINES` to allow skipping empty rows about jackson-dataformats-text HOT 17 CLOSED

For documentation purposes, I'll note that feature itself was actually added in 2.9, but implementation not. This is unfortunate.

from jackson-dataformats-text.

And there was much rejoining for now @vboulaye's patch is merges into 2.10, and the feature shall be supported in patch release 2.10.1!

from jackson-dataformats-text.

Is anyone working on this? I would be happy to pick it up.

from jackson-dataformats-text.

Quick note: I think this makes more sense as CsvParser.Feature than CsvSchema setting since it's not as much a property of document than handling (although I can see why someone might argue otherwise too). But I think this is more convenient from user perspective.

from jackson-dataformats-text.

note: Original had 5 thumbs-up, couple of +1s. So highly requested.

from jackson-dataformats-text.

@kg29 No one is workign on this as far as I know, so help would be highly valued!

from jackson-dataformats-text.

This is the only thing preventing us from switching from commons-csv.

from jackson-dataformats-text.

@kunickiaj One thing that could help (I may have time to look into this after completing non-blocking json parser) would be a failing unit test that simply shows how you'd like this to work. That is, test that would pass after fix.

from jackson-dataformats-text.

Yeah, I may look into this once I get a free moment.

We did some benchmarks against jackson csv + guard clause on our side vs commons-csv and it pretty much wiped out any performance gains from moving to jackson.

Figure fixing it here should be a much better option.

from jackson-dataformats-text.

Here's a trivial test which would pass given the desired behavior.

https://github.com/kunickiaj/jackson-dataformats-text/commit/b34aa58b5cb0a6926485d70fc17d2bc296eac925

from jackson-dataformats-text.

Excellent thank you; I'll have a look.

from jackson-dataformats-text.

Hmmh. Looking at code, this may not be easy to implement. As things are, START_OBJECT (or, without schema, START_ARRAY) is returned without looking at line. Logic would need to check for this setting, then do look-ahead... maybe that works.

from jackson-dataformats-text.

Hi @cowtowncoder how did this go? Has this been implemented? Your last comment was a wee bit confusing.
Cheers,
Ed

from jackson-dataformats-text.

@ehills No, functionality has not been added and does not exist.

from jackson-dataformats-text.

Maybe I misunderstand something here, but from the description in the first comment, I think this should be described as "skip blank lines", i.e. lines with only whitespace. Maybe you'd like to add SKIP_BLANK_LINES instead of changing SKIP_EMPTY_LINES to do this, otherwise code which uses the library could change its behaviour when using the new version.

Also, in the linked unit test I don't find any blank (only whitespace) lines, just empty lines:

Current CSV string:
final String CSV = "1,"xyz"\n\ntrue,\n";

Test for blank line being removed would need something like:
final String CSV = "1,"xyz"\n\ntrue,\n \n";

The rest of the test would be identical.

Cheers, Anders

from jackson-dataformats-text.

Not works for me yet

from jackson-dataformats-text.

Good point on "skip blank" vs "skip empty".

from jackson-dataformats-text.