Question: where to include "meta" statements (`account...`, `commodity format` & `alias`) & prices? about hledger-flow HOT 13 CLOSED

Hi @sbibauw, thanks for mentioning these issues. I think I have answers to at least some of them, and we should probably update the README with additional info.

About the "meta" statements you mention, this is a glaring gap in the docs:
you can add a directives.journal at the top-level directory, and the generated all-years.journal should then include it.

$ cat all-years.journal
### Generated by hledger-flow - DO NOT EDIT ###

!include directives.journal
!include import/all-years.journal

I initially added this as part of #50, but left the issue open in case it doesn't solve the issue. There hasn't been further activity on that issue, so I think we can close it and document the directives.journal.

My LEDGER_FILE environment variable is also pointed this top-level all-years.journal. I think this part is mentioned in the README though.

I see you have prices in your meta file. Prices are included separately, and described here:
https://github.com/apauley/hledger-flow#price-files

Can you expand a bit more on the decimal mark issue? I think I've used some preprocess scripts to change the decimal in some files I have, but maybe there's a better way.

from hledger-flow.

I'll have to play around with this. Maybe I can include directives.journal as a parameter to hledger when it generates the transactions.

@sbibauw Would you mind trying out the 0.15 release on the Paypal files you mentioned to see if the amount format is now different? I've switched to using hledger import, and I'm giving it the directives.journal as a parameter, so it should use the commodity formats you declared.

from hledger-flow.

I can't help pointing out that !include is ancient deprecated syntax - the ! Is unnecessary. Just in case you weren't aware. :)

@simonmichael The latest release now generates files without the obsolete !include syntax :-)

from hledger-flow.

@sbibauw Would you mind trying out the 0.15 release on the Paypal files you mentioned to see if the amount format is now different? I've switched to using hledger import, and I'm giving it the directives.journal as a parameter, so it should use the commodity formats you declared.

Yes, it works! I had solved my Paypal issue, but I just noticed today I had this issue in another file, and YES, the 0.15 release changes the amount format in the generated journal files! Thanks a lot for that!
And the use of hledger import works great with other things too, by putting explicit amounts when omitted for amount2, by using right placement of currency (following directives), etc. So, great!

from hledger-flow.

An additional issue linked to this: today, I tried to import a CSV which has a different decimal-mark (,) than the main journal (.). I saw that hledger knows how to convert to the default commodity format when it imports, but because hledger flow import makes hledger importing it to a file that has none of those meta statements, it doesn't convert it, which is causing a lot of trouble.

from hledger-flow.

from hledger-flow.

I can't help pointing out that !include is ancient deprecated syntax - the ! Is unnecessary. Just in case you weren't aware. :)

Thanks, I'll have to update the include file generation to use less ancient syntax 😃

from hledger-flow.

Thank you for the detailed answer @apauley . directives.journal works great. I didn't catch the location of prices/2021/journal.prices, so thanks for pointing that out. (I really believe that putting a more detailed example on hledger-flow-example would help make it a lot clearer for new users: I'll submit a PR there.)

So, there are only 2 things that I'm still wondering:

1. Is there a way to make use of hledger add appropriately, so that manually added transactions will go to some import/owner/_manual_/2021/post-import.journal? With the current workflow, they are added to ./all-years.journal and get erased at every run of hledger-flow!

2. The decimal mark issue: I have a Paypal CSV with comas as decimal separator:

    "Date","Time","TimeZone","Name","Type","Status","Currency","Gross","Fee","Net","Receipt ID","Balance","Tip"
    "01/06/2021","08:05:16","PDT","The New York Times Co.","PreApproved Payment Bill User Payment","Completed","EUR","-1,00","0,00","-1,00","","10,00",""
   

   The rules file contains

    decimal-mark ,
   

   and that works to make sure hledger imports it to the journal appropriately:

    2021-06-01 The New York Times Co
        assets:Paypal:eur                €-1,00 = €10,00
        expenses:press                    €1,00
   

   BUT as you can see, the coma decimal mark remains, which causes some problems down the line.
   Now, hledger is actually supposed to modify the decimal mark to match the one set in the commodity format directive, that I've now put in directives.journal:

    commodity 
      format  1,000.00 $
   

   but in the current hledger-flow workflow, at the moment of importing the CSV applying the rules, hledger is unaware of the directives.journal containing this format and so it does not convert the decimal mark.

   Now, I must say that, while I think that it would be great to implement that somehow, for ideal design, I have now found a solution: using the paypalcsv gist by @simonmichael to import Paypal transactions via the Paypal API, the CSV is correctly formatted with a more standard decimal point.

from hledger-flow.

from hledger-flow.

Nice, could I ask how you found that ? I had forgotten it.

I'm pretty sure I found the link to this gist somewhere on your doc site for hledger, but now, I've been trying to find it without any luck!

from hledger-flow.

So, there are only 2 things that I'm still wondering:

1. There isn't a built-in way to easily or automatically let the tool choose the manual journal file. Personally I only add transactions manually in a few cases, and then I specify the file I want with my own hands like a cave-man.

In my case the _manual_/YYYY/*-import.journal just contains more include statements which I use to further organize my journals. So for me personally I don't think it will help that much if a tool picks a *-import.journal file for me. Maybe it can help if the tool is smart enough to auto-complete files based on the mentioned include statements. You'll probably use it in different ways though. If you find that this is a place of improvement that will save you lots of time, maybe start thinking about ways you would want to use it.

2. I'll have to play around with this. Maybe I can include directives.journal as a parameter to hledger when it generates the transactions. I see there are more imrovements similar to this one which I haven't had a chance to look at yet: #97

from hledger-flow.

Thank you for these answers

For hledger add and hledger-iadd, @simonmichael kindly pointed to the fact that hledger accepts multiple -f arguments and will add the transactions to the first file. So the solution for adding manual transactions while taking all the history & directives into account is to do something like:

hledger add -f import/_manual_/2021/post-import.journal -f all-years.journal

I understand that, if you can, it is obviously better not to have to add many manual transactions, but I live in a country where many transactions (including important ones, like buying a car) are still often done in cash. Hence the need for it.
I'm not so convinced by the naming of the "pre/post-import.journal" in the case of cash transactions (because they are not pre/post), but that's unimportant.

from hledger-flow.

I didn't realize that this was a partial duplicate of #50 . Sorry!

from hledger-flow.