Quoted delimiters still interpreted about rainbow_csv HOT 6 CLOSED

Hi @fonsecapeter ,
Thank you for interest in this small project!

About your suggestion, I think this is a bad idea, and here I will try to explain why.

1. The only widespread delimiter that can be quoted is comma. I have no proof of the statement, but I have never seen a quoted tab in tsv. Maybe practice of quoting semicolons also exist, but I've never seen such data tables. Also semicolons are rarely used as delimiters, so quoted semicolons must be even more exotic.

2. csv with escaped (or quoted) commas is a terrible format. I hope you can agree with this statement too. So the first thing an experienced user will do when he sees a quoted csv file, he will try to somehow convert it to tsv or json using some conversion tool. And here is where rainbow_csv will help - when highlighting is not working or looks broken for some lines in the file, it will encourage user to look more closely at the file they have, and after realizing it has escaped commas, quickly convert it to a more decent format (tsv or json or anything). So I can say that the broken highlighting is not a bug - it is a feature that helps to detect these awful files with escaped commas.

I am still open to the further discussion of this very interesting subject. A cool experiment IMO would be to crawl github with some smart web crawler and see what percentage of *.csv files have escaped commas.

from rainbow_csv.

Thanks @mechatroner,

You're totally right -- could always just reformat the file (which should usually happen anyway) and this could be a nice feature to make it super easy to spot before any other csv data loaders trip over it.

I was having some distant memories to using python3's builtin csv package, which can take a quote_char and a delimiter -- but I think that gets more into *sv territory instead of pure csv.

Might be interesting to see what kind of weird things are happening in github csv's/tsv's in general and in what proportions (could probably inform some kind of standardization too)... will let you know if I ever get around to doing something like that 😓

Anyway keep up the good work!

from rainbow_csv.

@fonsecapeter , I recently realized that quoted csv is more popular format than I thought. And I already wrote some code to implement escaped highlighting in "quoted_hl" branch. I will try to find any counterexamples that can break the new regexps, but if everything is OK it will be soon merged into master. Thank you very much for this suggestion, now I think that this is a Cool Idea =)

from rainbow_csv.

Awesome, thanks!

Just saw your new queries feature too, this is definitely the coolest way to work with spreadsheets in vim

from rainbow_csv.

csv highlighting is rfc-compliant now https://tools.ietf.org/html/rfc4180

from rainbow_csv.

Maybe I've closed this too early, because what I implemented is double quote escaping variant, but as you said there is also backslash quote escaping variant e.g.
this quote escaping works now:
12,"hello, ""world""!",100
and this doesn't:
12,"hello, \"world\"!",100
So, @fonsecapeter , if you still want to help, you can implement this quoting policy, we need both syntax rules and python, js and vim-script splitting functions.

from rainbow_csv.