`hledger: user error CSVError (ParseErrorBundle...` is rather unclear about hledger-flow HOT 8 CLOSED

The error happens even if I ensure there are only ASCII characters in the CSV file. So this is not an encoding issue.

$ vim bank/account/preprocess
iconv -f iso-8859-1 -t ASCII//TRANSLIT "$1" > "$2"

from hledger-flow.

I believe this is caused by the use of the semicolon separator (as opposed to comma) in the CSV file, but the error does not give any hint as to what separator is in use, which hinders troubleshooting for first time users.

Beyond that, it's not obvious how to override the hledger import's --separator option. And .rules file do not support specifying this either. A possible workaround is changing all semicolons into commas within the preprocess script. Not ideal, but feasible.

Relevant issues:

• simonmichael/hledger#176 (need for configurable separator, closed after the addition of --separator to hledger import)
• simonmichael/hledger#868 (need for configurable separator support in .rules files, still open)

from hledger-flow.

In the short term I would suggest using your existing preprocess script to change the separator.
The preprocess script is intended to manipulate the format of CSV files, I think this is within the scope of that.

In the long term this would be solved in simonmichael/hledger#868 like you said.
Maybe you could give your opinion on the whitespace separator issue under discussion, and by doing so assist in the completion if that issue?

from hledger-flow.

This issue is more about the clarity of the error message. If the clarity of the message is outside of hledger-flow control, then I need to close this issue and take it up to them. Let me know.

from hledger-flow.

@lestephane We can improve our error messages to be as clear as possible as to what is failing and if it is hledger-flow or an external program that failed.

But there has been a lot of improvements after 0.11.1.2. Can you try it using 0.12?

I've changed the separators in one of the example input files, and this is what the error looks like in 0.12:

$ hledger-flow --version
hledger-flow 0.12.0.0
$ hledger-flow import
Collecting input files...
Found 3 input files in 0.027027743s. Proceeding with import...
hledger: user error (CSVError (ParseErrorBundle {bundleErrors = TrivialError 0 (Just EndOfInput) (fromList []) :| [], bundlePosState = PosState {pstateInput = "", pstateOffset = 0, pstateSourcePos = SourcePos {sourceName = "/home/andreas/Code/Hledger/hledger-flow-example/import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-03-30.csv", sourceLine = Pos 1, sourceColumn = Pos 1}, pstateTabWidth = Pos 8, pstateLinePrefix = ""}}))

=== Begin Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-03-30.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===
External command:
/usr/bin/hledger print --rules-file /home/andreas/Code/Hledger/hledger-flow-example/import/gawie/bogart/cheque/bogart-cheque.rules --file /home/andreas/Code/Hledger/hledger-flow-example/import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-03-30.csv --output-file /home/andreas/Code/Hledger/hledger-flow-example/import/gawie/bogart/cheque/3-journal/2016/123456789_2016-03-30.journal
Exit code 1
Error output:
hledger: user error (CSVError (ParseErrorBundle {bundleErrors = TrivialError 0 (Just EndOfInput) (fromList []) :| [], bundlePosState = PosState {pstateInput = "", pstateOffset = 0, pstateSourcePos = SourcePos {sourceName = "/home/andreas/Code/Hledger/hledger-flow-example/import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-03-30.csv", sourceLine = Pos 1, sourceColumn = Pos 1}, pstateTabWidth = Pos 8, pstateLinePrefix = ""}}))

=== End Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-03-30.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===

I'm keen to hear any suggestions you have on making our error messages clearer.

Although I don't think I would like to try and interpret the output of external programs to make their messages clearer, for that I'd prefer to go upstream.

from hledger-flow.

@lestephane Did you have a look at how the errors are displayed in the latest version? If you think they are fine then we can close this issue, otherwise I'd love to hear your thoughts on how we can improve it.

from hledger-flow.

Let me try just now

from hledger-flow.

The clarity of the error message is not something hledger-flow can do much about, but in the latest version it does at least include the command line that caused the error, which is sufficient for my purpose.

I've since moved away from csv rules file, using mostly awk based construct and preprocess scripts to be fully in control, but the improved logging will no doubt help others. Thanks!

from hledger-flow.