Code Monkey home page Code Monkey logo

csv2ofx's Introduction

csv2ofx

travis versions pypi

INTRODUCTION

csv2ofx is a Python library and command line interface program that converts CSV files to OFX and QIF files for importing into GnuCash or similar financial accounting programs. csv2ofx has built in support for importing csv files from mint, yoodlee, and xero.

Requirements

csv2ofx has been tested and is known to work on Python 3.7, 3.8, and 3.9; and PyPy3.7.

INSTALLATION

(You are using a virtualenv, right?)

sudo pip install csv2ofx

Usage

csv2ofx is intended to be used either directly from Python or from the command line.

Library Examples

normal OFX usage

import itertools as it

from meza.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.ofx import OFX
from csv2ofx.mappings.default import mapping

ofx = OFX(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])

for line in IterStringIO(content):
    print(line)

normal QIF usage

import itertools as it

from tabutils.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.qif import QIF
from csv2ofx.mappings.default import mapping

qif = QIF(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = qif.gen_groups(records)
trxns = qif.gen_trxns(groups)
cleaned_trxns = qif.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([qif.gen_body(data), qif.footer()])

for line in IterStringIO(content):
    print(line)

CLI Examples

show help

csv2ofx -h

usage: csv2ofx [options] <source> <dest>

description: csv2ofx converts a csv file to ofx and qif

positional arguments:
  source                the source csv file (default: stdin)
  dest                  the output file (default: stdout)

optional arguments:
  -h, --help            show this help message and exit
  -a TYPE, --account TYPE
                        default account type 'CHECKING' for OFX and 'Bank' for QIF.
  -e DATE, --end DATE   end date (default: today)
  -B BALANCE, --ending-balance BALANCE
                        ending balance (default: None)
  -l LANGUAGE, --language LANGUAGE
                        the language (default: ENG)
  -s DATE, --start DATE
                        the start date
  -y, --dayfirst        interpret the first value in ambiguous dates (e.g. 01/05/09) as the day
  -m MAPPING_NAME, --mapping MAPPING_NAME
                        the account mapping (default: default)
  -x FILE_PATH, --custom FILE_PATH
                        path to a custom mapping file
  -c FIELD_NAME, --collapse FIELD_NAME
                        field used to combine transactions within a split for double entry statements
  -C ROWS, --chunksize ROWS
                        number of rows to process at a time (default: 2 ** 14)
  -r ROWS, --first-row ROWS
                        the first row to process (zero based)
  -R ROWS, --last-row ROWS
                        the last row to process (zero based, negative values count from the end)
  -O COLS, --first-col COLS
                        the first column to process (zero based)
  -L, --list-mappings   list the available mappings
  -V, --version         show version and exit
  -q, --qif             enables 'QIF' output instead of 'OFX'
  -M, --ms-money        enables MS Money compatible 'OFX' output
  -o, --overwrite       overwrite destination file if it exists
  -D DATE, --server-date DATE
                        OFX server date (default: source file mtime)
  -E ENCODING, --encoding ENCODING
                        File encoding (default: utf-8)
  -d, --debug           display the options and arguments passed to the parser
  -v, --verbose         verbose output

normal usage

csv2ofx file.csv file.ofx

print output to stdout

csv2ofx ~/Downloads/transactions.csv

read input from stdin

cat file.csv | csv2ofx

qif output

csv2ofx -q file.csv

specify date range from one year ago to yesterday with qif output

csv2ofx -s '-1 year' -e yesterday -q file.csv

use yoodlee settings

csv2ofx -m yoodlee file.csv

Special cases

Some banks, like UBS Switzerland, may provide CSV exports that are not readily tractable by csv2ofx because of extra header or trailing lines, redundant or unwanted columns. These input files can be preprocessed with the shipped utilz/csvtrim shell script. F.i., with mapping ubs-ch-fr:

csvtrim untrimmed.csv | csv2ofx -m ubs-ch-fr

CUSTOMIZATION

Code modification

If you would like to import csv files with field names different from the default, you can modify the mapping file or create your own. New mappings must be placed in the csv2ofx/mappings folder (otherwise you must use the ). The mapping object consists of a dictionary whose keys are OFX/QIF attributes and whose values are functions which should return the corresponding value from a record (csv row). The mapping function will take in a record, e.g.,

{'Account': 'savings 2', 'Date': '1/3/15', 'Amount': 5000}

The most basic mapping function just returns a specific field or value, e.g.,

from operator import itemgetter

mapping = {
    'bank': 'BetterBank',
    'account': itemgetter('Account'),
    'date': itemgetter('Date'),
    'amount': itemgetter('Amount')}

But more complex parsing is also possible, e.g.,

mapping = {
    'account': lambda r: r['Details'].split(':')[0],
    'date': lambda r: '%s/%s/%s' % (r['Month'], r['Day'], r['Year']),
    'amount': lambda r: r['Amount'] * 2,
    'first_row': 1,
    'last_row': 10,
    'filter': lambda r: r['Amount'] > 10,
}

Required field attributes

attribute description default field example
account transaction account Account BetterBank Checking
date transaction date Date itemgetter('Transaction Date')
amount transaction amount Amount itemgetter('Transaction Amount')

Optional field attributes

attribute description default field default value example
desc transaction description Reference n/a shell station
payee transaction payee Description n/a Shell
notes transaction notes Notes n/a for gas
check_num the check or transaction number Row n/a 2
id transaction id check_num Num n/a
bank the bank name n/a account Bank
account transaction account type n/a checking savings
account_id transaction account id n/a hash of account bb_checking
type transaction type (either debit or credit) n/a CREDIT if amount > 0 else DEBIT debit
balance account balance n/a n/a $23.00
class transaction class n/a n/a travel

Optional value attributes

attribute description default value example
has_header does the csv file have a header row True
custom_header header row to use (e.g. if not provided in csv) None ["Account","Date","Amount"]
is_split does the csv file contain split (double entry) transactions False
currency the currency ISO code USD GBP
delimiter the csv field delimiter , ;
date_fmt custom QIF date output format %m/%d/%y %m/%d/%Y
dayfirst interpret the first value in ambiguous dates (e.g. 01/05/09) as the day (ignored if parse_fmt is present) False True
parse_fmt transaction date parsing format %m/%d/%Y
first_row the first row to process (zero based) 0 2
last_row the last row to process (zero based, negative values count from the end) inf -2
first_col the first column to process (zero based) 0 2
filter keep transactions for which function returns true lambda tr: tr['amount'] > 10

Scripts

csv2ofx comes with a built in task manager manage.py.

Setup

pip install -r dev-requirements.txt

Examples

Run python linter and nose tests

manage lint
manage test

Contributing

Please mimic the coding style/conventions used in this repo. If you add new classes or functions, please add the appropriate doc blocks with examples. Also, make sure the python linter and nose tests pass.

Ready to contribute? Here's how:

  1. Fork and clone.
git clone [email protected]:<your_username>/csv2ofx.git
cd csv2ofx
  1. Setup a new virtualenv
mkvirtualenv -i pkutils csv2ofx
activate csv2ofx
python setup.py develop
pip install -r dev-requirements.txt
  1. Create a branch for local development
git checkout -b name-of-your-bugfix-or-feature
  1. Make your changes, run linter and tests (see above), and submit a pull request through the GitHub website.

Adding Mappings

How to contribute a mapping:

  1. Add the mapping in csv2ofx/mappings/
  2. Add a simple example CSV file in data/test/.
  3. Add the OFX or QIF file that results from the mapping and example CSV file in data/converted/.
  4. Add a csv2ofx call for your mapping to the tests in tests/test.py, in PRE_TESTS. If you added an OFX (not QIF) converted file, pay attention to the -e (end date) and -D (server date) arguments in the test- otherwise tests may pass on your workstation and fail on the build server.
  5. Ensure your test succeeds (see above).

License

csv2ofx is distributed under the MIT License, the same as meza.

csv2ofx's People

Contributors

ak2k avatar alper avatar axisnl avatar brunobasanta avatar craftsman95 avatar d-lord avatar jaraco avatar keystrike avatar kovah avatar mattfox avatar mypalmike avatar pcr20 avatar pricechild avatar pyup-bot avatar rbuccigrossi avatar reubano avatar sheamuspatt avatar sphakka avatar sshort avatar tstabrawa avatar xcambar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

csv2ofx's Issues

Add option to specify date format. Also add message to inform user of hidden transactions.

The following archive contains a mapping function: bpn.py which works with BNP Paribas Fortis CSV files and a csv file containing data that are not properly parsed by csv2ofx.

problem_parsing_csv.zip

When bpn.py is properly installed, csv2ofx is launched with the following command:

$ csv2ofx -m bnp -q  problem.csv
!Account
NBNP courant
TBank
^
!Type:Bank
D01/01/01
Pfoo
Mbar
T-500000.00
^

Only the first line is added. It seems that the date ('Date valeur', the third column) cannot be parsed but I cannot understand why.

$ cat problem.csv 
sequence,plop,Date valeur,Montant,Devise du compte,CONTREPARTIE DE LA TRANSACTION,Details,Numero
2018-1515,01/01/01,01/01/01,-500000,EUR,foo,bar,42
2018-0049,12/02/2018,12/02/18,-691.82,EUR,foo,bar,43
2018-0048,08/02/2018,08/02/18,-12.5,EUR,foo,bar,44
2018-0046,07/02/2018,07/02/18,-16.2,EUR,foo,bar,45
2018-0045,06/02/2018,06/02/18,-353.89,EUR,foo,bar,46
2018-0044,05/02/2018,04/02/18,-150,EUR,foo,bar,47

As the csv is parsed with meza.io, it uses generator that I don't know how to debug.

Thanks,

Transactions skipped due to date

I'm trying to use csv2ofx in python as a library. I'm using the "normal OFX usage" example code found in the README. However, none of the transactions are getting exported to OFX. I narrowed it down to this line in the "clean_trxns" function:
filtered_trxns = it.ifilterfalse(self.skip_transaction, trxns)

It seems as though they're being skipped because they don't fall within the right dates. Is there a clean way to set the start/end dates when using as a library?

errors: either 'field_0 is missing from file' or 'Parser error'

Hi,

I'm trying to get csv2ofx to work.
I tried to create a (very simple) custom mapping but nothing seems to work.

Version of csv2ofx:
(csv2ofx)me@linux-fsnh:~/temp> csv2ofx -V
v0.22.0

Version of python: 3.5.4

input csv file:
(csv2ofx)me@linux-fsnh:~/temp> cat dummy.csv
BE90 1111 2222 3333 , 449.8 , 27-12-2017 , Test

mapping file:


from operator import itemgetter

mapping = {
    'has_header': False,
    'is_split': False,
    'bank': 'Bank',
    'currency': 'EUR',
    'delimiter': ',',
#    'account': itemgetter('field_0'),
    'account': 'test',
    'date': itemgetter('field_2'),
    'amount': itemgetter('field_1'),
    'desc': itemgetter('field_3'),
}

Either if I try with

'account': itemgetter('field_0'),

then following error is shown:

Field 'field_0' is missing from file. Check `mapping` option.

If I try with

'account': 'test',

then follwing error is shown:

(csv2ofx)me@linux-fsnh:~/temp> csv2ofx -m dummy dummy.csv 
Parser must be a string or character stream, not NoneType
No data to write. Parser must be a string or character stream, not NoneType. Try again with `-c` option.

Any help to point out what is going wrong is appreciated.
Thx.

br,
Ruben

Support investment transactions

My HSA provider doesn't support Quicken export anymore. It does support Excel export, however. I was hoping to use this tool to bridge the gap, but unfortunately, this tool doesn't seem to support investment transactions (<INVSTMTMSGSETV1> in OFX parlance).

Sell and Buy investment transactions based on positive or negative amount.

Thanks for an awesome script - got most of it working for an ANZ investment statement. However I can't figure out how to get the transaction type to be Sell for negative GrossAmount and Buy for positive GrossAmount.
I'm also not sure if I got the mapping correct as the documentation is very sparse for investement statements.

Here's the CSV
Fund,EffectiveDate,Description,GrossAmount,NetAmount,UnitPrice,Units ANZ Growth Fund ,9/03/2018,PIE Rebate,0.95,0.95,1.91,0.4966,598.6836 ANZ Growth Fund ,2/03/2018,Administration fee,-2.00,-2.00,1.90,1.0513 ANZ Growth Fund ,2/02/2018,Administration fee,-2.00,-2.00,1.95,1.0232 ANZ Growth Fund ,4/01/2018,Administration fee,-2.00,-2.00,1.94,1.0301 ANZ Growth Fund ,4/12/2017,Administration fee,-2.00,-2.00,1.91,1.0456 ANZ Growth Fund ,3/11/2017,Administration fee,-2.00,-2.00,1.89,1.0607 ANZ Growth Fund ,6/10/2017,Administration fee,-2.00,-2.00,1.85,1.079 ANZ Growth Fund ,8/09/2017,Administration fee,-2.00,-2.00,1.82,1.0981 ANZ Growth Fund ,4/08/2017,Administration fee,-2.00,-2.00,1.80,1.1113

My Mapping
mapping = { 'bank': 'ANZ' ,'has_header': True ,'is_split': False ,'is_investment': True ,'account_types': 'Invst' ,'currency': 'NZD' ,'date_fmt': '%m/%d/%Y' ,'date': itemgetter('EffectiveDate') ,'amount': itemgetter('GrossAmount') ,'price': itemgetter('UnitPrice') ,'shares': itemgetter('Units') ,'account': itemgetter('Fund') ,'symbol': itemgetter('Fund') ,'investment': itemgetter('Fund') ,'ticker': itemgetter('Fund') ,'desc': itemgetter('Description') ,'delimiter':',' ,'type': 'buy' }

And the output
!Account NANZ Growth Fund TNone ^ !Type:None D09/03/2018 NShrsIn YANZ Growth Fund I1.91 Q0.4966 Cc MPIE Rebate T0.95 ^ !Type:None D02/03/2018 NShrsIn YANZ Growth Fund I1.90 Q1.0513 Cc MAdministration fee T2.00 ^ !Type:None D02/02/2018 NShrsIn YANZ Growth Fund I1.95 Q1.0232 Cc MAdministration fee T2.00 ^ !Type:None D04/01/2018 NShrsIn YANZ Growth Fund I1.94 Q1.0301 Cc MAdministration fee T2.00 ^ !Type:None D04/12/2017 NShrsIn YANZ Growth Fund I1.91 Q1.0456 Cc MAdministration fee T2.00 ^ !Type:None D03/11/2017 NShrsIn YANZ Growth Fund I1.89 Q1.0607 Cc MAdministration fee T2.00 ^ !Type:None D06/10/2017 NShrsIn YANZ Growth Fund I1.85 Q1.079 Cc MAdministration fee T2.00 ^ !Type:None D08/09/2017 NShrsIn YANZ Growth Fund I1.82 Q1.0981 Cc MAdministration fee T2.00 ^ !Type:None D04/08/2017 NShrsIn YANZ Growth Fund I1.80 Q1.1113 Cc MAdministration fee T2.00 ^

Ideas for improving the documentation

I played with this library today, hoping it'd help me convert my financial data into a format I could upload to QuickBooks. Unfortunately, QuickBooks rejects the output files, with unhelpful error messages, even though GNUCash does them.

In any case, some improvement to the README.md documentation would've helped me understand this library a little more quickly. Here are some suggestions:

  • Specify that the expected date format is MM/DD/YYYY—this is an unusual format (only two or three countries use it), so it ought to be made clear
  • Mention that, by default, future-dated transactions are excluded (some of my transactions were missing because they were from the other side of the international dateline)
  • Explain how mapping works. e.g. 'mapping should be a map of output field names to functions that return values for those fields. The functions should expect a single parameter: a map of input field names to their values. For example, { 'date': lambda r: '%s/%s/%s' % (r['month'], r['day'], r['year']) } expects the input CSV to have separate "month", "day" and "year" columns, and will combine these into a single "date" output column, in the MM/DD/YYYY format.'
  • For each field, particularly the optional ones, specify, which OFX elements they map to, e.g. that notes and class get combined into the <MEMO> element
  • Explain how splits work.

Documentation: How do I get <LEDGERBAL> in ofx file?

If i want a LEDGERBAL in my footer how would I get this? In CLI we can do a test file csv2ofx -m capitalone capitalone.csv cap.ofx . In the code I can see that some split is needed, so I assumed -S as it would be in the documentation listed on github but it is not present when doing csv2ofx -h .

Rabobank update

I don't even know how to create a pull request, but the rabobank (Dutch bank) updated it's format. Here's the new mapping.
rabobank.py.txt

account mapping seems to have issues

Hi @reubano. I tried creating my own mapping for the Dutch Rabobank, but ran into an issue where csv2ofx always errors out with Field 'field_0' is missing from file. Check 'mapping' option.
This actually seems to be related to the account key in the mapping dict and not with the field_0. When I assign a static string to account it errors out the same error just with Field 'somestring' is missing from file. Check 'mapping' option.
I actually get the same Field 'field_0' is missing from file. Check 'mapping' option. when I use the included abnamro mapping.

Parsing dates incorrectly

Doesn't seem to work correctly when csv has dates in %d.%m.%Y, which is rather common format outside of the US.

When %d > 12, it does save the correct date to the output.
When %d ≤ 12, it switches the day+month.

08.01.2019 => 20190801000000 # Incorrect
14.01.2019 => 20190114000000 # Correct
06.02.2019 => 20190602000000 # Incorrect
14.02.2019 => 20190214000000 # Correct
06.03.2019 => 20190603000000 # Incorrect
06.03.2019 => 20190603000000 # Incorrect
18.03.2019 => 20190318000000 # Correct
06.04.2019 => 20190604000000 # Incorrect

Tried fixing it on my own, but don't quite understand how parsing the dates is built here.

Referencing columns using column_# does not work when csv2ofx is invoked with the Python library

I've been using csv2ofx successfully for some time with some custom mapping files and a bash script which runs cs2ofx using the CLI interface.

To go cross platform I re-wrote the bash script in Python and replaced the CLI with the Python library. However now, any banks with mapping files that use column_# to reference the columns (because the CSV has no header) now fail. Either the script fails with an error, or the field is just blank.

An example mapping file using column_# which now fails is:

from __future__ import (
    absolute_import, division, print_function, unicode_literals)

from operator import itemgetter

# Convert DD/MM/YYYY date to MM/DD/YYYY (the format required for csv2ofx)
def date_func(tr):
    date = tr['column_1']
    return '{}/{}/{}'.format(date[3:5], date[:2], date[-4:])

# Remove additional spaces
def remove_spaces(text):
    return " ".join(text.split())

mapping = {
    'has_header': False,
    'is_split': False,
    'bank': 'Citibank',
    'currency': 'AUD',
    'delimiter': ',',
    'date': date_func,
    'payee': lambda tr: remove_spaces(tr['column_2']),
    'amount': itemgetter('column_3'),
    'account': itemgetter('column_5')
}

The source data for this mapping looks like:

"03/07/2019","Miscellaneous BOOTS 2137             GATWICK       GBR","-8.13","","'123456789'"
"03/07/2019","Miscellaneous NEW SOUTHERN RAILW     HOVE 4038     GBR","-19.56","","'123456789'"

I invoke csv2ofx using the library and write the file to disk like so:

ofx = OFX(mapping)
records = read_csv(source_file)
groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])

# Save the file to disk
dest_file = source_file + ".ofx"
df = open(dest_file, "wb")
for line in IterStringIO(content):
    df.write(line)
    # print(line)
df.close()

The error response for the example above is:

Traceback (most recent call last):
  File ".\csv2ofx-simon.py", line 170, in <module>
    for line in IterStringIO(content):
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 113, in __next__
    return self._read(next(self.lines))
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 123, in <genexpr>
    return (g for k, g in groups if k)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 66, in <genexpr>
    encode = lambda iterable: (s.encode(ENCODING) for s in iterable)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\ofx.py", line 444, in gen_body
    for datum in data:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\utils.py", line 152, in gen_data
    for group, main_pos, sorted_trxns in groups:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\__init__.py", line 294, in clean_trxns
    for grp, trxns in groups:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\__init__.py", line 280, in gen_trxns
    for grp, transactions in groups:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\ofx.py", line 484, in gen_groups
    for gee in group(cleansed, keyfunc):
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\process.py", line 591, in group
    sorted_records = sorted(records, key=keyfunc)
KeyError: 'column_5'

The error response does vary depending on the first field to fails. For example, if the account name is hard coded with 'account': 'Citibank', the error changes to:

Traceback (most recent call last):
  File ".\csv2ofx-simon.py", line 170, in <module>
    for line in IterStringIO(content):
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 113, in __next__
    return self._read(next(self.lines))
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 123, in <genexpr>
    return (g for k, g in groups if k)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\meza\io.py", line 66, in <genexpr>
    encode = lambda iterable: (s.encode(ENCODING) for s in iterable)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\ofx.py", line 444, in gen_body
    for datum in data:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\utils.py", line 152, in gen_data
    for group, main_pos, sorted_trxns in groups:
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\__init__.py", line 314, in clean_trxns
    sorted_trxns = sorted(enumerate(filtered_trxns), key=keyfunc)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\csv2ofx\__init__.py", line 161, in skip_transaction
    return not self.end >= parse(self.get('date', trxn)) >= self.start
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\dateutil\parser\_parser.py", line 1358, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\dateutil\parser\_parser.py", line 646, in parse
    res, skipped_tokens = self._parse(timestr, **kwargs)
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\dateutil\parser\_parser.py", line 722, in _parse
    l = _timelex.split(timestr)         # Splits the timestr into tokens
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\dateutil\parser\_parser.py", line 207, in split
    return list(cls(s))
  File "C:\Users\simon\AppData\Local\Programs\Python\Python37\lib\site-packages\dateutil\parser\_parser.py", line 76, in __init__
    '{itype}'.format(itype=instream.__class__.__name__))
TypeError: Parser must be a string or character stream, not NoneType

I've run out of ideas of how to solve this (other than by using the CLI instead of the Python library), so if anyone is able to help that would be great!

Thanks

QuickBooks to OFX Problems

So in QuickBooks 2007, I found a range of transactions and exported them by generating a report and exporting them to Excel which I saved to a .csv. In emacs, I then removed the first few lines of the report to get a file that starts with
,,Type,Date,Num,Name,Memo,Account,Clr,Split,Amount,Balance
and each transaction follows (the first field is always empty.

When I run csv2ofx, I'm greeted with a the help text. If I change 'Account' above to 'Fred' then I get an error that it can't find the Account field.

Any ideas what might be going amiss? Or even where to start tacking things down since there are no helpful messages at all.

My ultimate goal, btw, is to migrate from QB2007 to gnucash...

"Reinvest" category interpreted as "Buy" instead of "ReinvDiv"

The category "reinvest" is converted to "Buy" instead of "ReinvDiv". This happens because "invest" is above "reinvest" in the ACTION_TYPES in utils.py (and therefore is matched first with your substring matching).

As an example, if you run the following input to the "mintapi" mapping to output qif:

account,odate,symbol,category,price,shares,amount,note
401k,06/02/2018,VAIX,reinvest,249.210785925682,3.041,757.85,EMPLOYEE 401(K)

The output action is "Buy" instead of "ReinvDiv".

I'm sending along a pull request to add a quick test and fix for this.

By the way, thank you for this project!

UnicodeDecodeError: 'utf8' codec can't decode

$ csv2ofx -ov -l PT-BR extrato-201609071200.csv extrato-201609071200.ofx
Traceback (most recent call last):
  File "/usr/local/bin/csv2ofx", line 20, in <module>
    run()
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/main.py", line 141, in run
    write(args.dest, IterStringIO(content), **kwargs)
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 691, in write
    return _write(filepath, content, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 679, in _write
    for c in ft.chunk(content, chunksize):
  File "/usr/local/lib/python2.7/site-packages/tabutils/fntools.py", line 248, in <genexpr>
    generator = (content.read(chunksize) for _ in it.count())
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 110, in read
    return self._read(self.iter, n)
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 103, in _read
    return ft.byte(it.islice(iterable, n)) if n else ft.byte(iterable)
  File "/usr/local/lib/python2.7/site-packages/tabutils/fntools.py", line 183, in byte
    tupled = tuple(content) if hasattr(content, 'next') else content
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 96, in <genexpr>
    return (s.encode(ENCODING) for s in iterable)
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/ofx.py", line 417, in gen_body
    for gd in data:
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/utils.py", line 130, in gen_data
    for group, main_pos, sorted_trxns in groups:
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/__init__.py", line 238, in clean_trxns
    for group, trxns in groups:
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/__init__.py", line 224, in gen_trxns
    for group, transactions in groups:
  File "/usr/local/lib/python2.7/site-packages/csv2ofx/ofx.py", line 451, in gen_groups
    for chnk in chunk(records, chunksize):
  File "/usr/local/lib/python2.7/site-packages/tabutils/fntools.py", line 263, in <genexpr>
    generator = (list(it.islice(i, chunksize)) for _ in it.count())
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 411, in read_csv
    for row in read_file(filepath):
  File "/usr/local/lib/python2.7/site-packages/tabutils/io.py", line 393, in read_file
    names = csv.reader(f, encoding=encoding, **kwargs).next()
  File "/usr/local/lib/python2.7/site-packages/unicodecsv/__init__.py", line 112, in next
    unicode_(value, encoding, encoding_errors)) for value in row]
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd3 in position 10: invalid continuation byte

sample file:

"DATA";"HISTÓRICO DE DESPESAS";"TIPO";"VALOR"
"10/Ago";"COMPRA DÉBITO NACIONAL - SOME         -SAO PAULO-BRA";"Débito";"- R$ 999,99"

QIF Date Output Hardcoded as US Style Date

Line 169 of qif.py:
kwargs.update({'time_stamp': kwargs['date'].strftime('%m/%d/%y')})

On my system I manually patched this to:
kwargs.update({'time_stamp': kwargs['date'].strftime('%d/%m/%Y')})

Suggestion:
Base output of QIF date on system locale, or include option in custom mapping file to choose output style.

Your tool has been very useful for me, thank you!

Thank you!

Thank you for your documentation for the QIF format! It is super hard to find documentation for it on the web, so I'm so glad I ran across yours!

Feel free to close this issue… I just wanted to thank you :-)

installation issues when installing with pip

Hello,
When I'm running the pip install command sudo pip install csv2ofx

I'm getting the following errors:

...

Installing collected packages: six, urllib3, idna, certifi, requests, python-dateutil, PyYAML, meza, csv2ofx
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-eaCbI3-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

Please advice
Sincerely,
Eran.

Account mapping does not work

Running example Python script for some Chase account activity CSV throws an error for mapped account:

mapping = {
    'bank': 'Chase',
    'account': 'Savings',
    'date': itemgetter('Post Date'),
    'amount': itemgetter('Amount'),
    'check_num': itemgetter('Check or Slip #'),
    'desc' : itemgetter('Description')
    }

Error: KeyError, 'Savings'

I tried to use any combination of account - always throw that error for whatever I specify in account property.

I'm testing it on Windows 7 x64 platform, using Anaconda python

CentOS7 support

Hello,

CentOS 7 contains python3.4. What mandates python3.5 in your project ? Could the requirements be lowered to python3.4 ?

Regards

Pascal

International currency support

On Sun, Sep 22, 2019, at 7:27 AM, Ottorino wrote:
Hi Reuben, here is Ottorino

I'm also "an open source enthusiast", since I strongly believe in 
cooperation rather than in competition.

I'm using git and github as a single user, but I'm not sure I'm able to 
fork and modify your repository: that's why I'm contacting by email

I'm a soil chemist teaching at the university, and the only programming 
language I'm proficient with is R.

I've found your csv2ofx a very valuable resource to convert my paypal 
and bank statements to import them in gnucash.

In particular, my local bank, is able to provide me  either an "amiable" 
xlsx file or a useless qif file.

I've been studying csv2ofx on github and then I wrote two bash scripts 
to manipulate the original files from paypal and from my bank 

I must thank you because on writing the scripts I was forced to push 
forward my (scarce) regexp knowledge. Writing those scripts has been 
challenging but formative.

The two scripts  are at your disposal and attached, just let me clean 
them up and translate the comments into English. Also the mappings are 
attached

I've started to use the scripts "on the field" and I realized that 
(probably) I've stepped on a bug: if the amounts are in the form 0,xx 
("," is our local decimal sign) csv2ofx throws errors.

To be sure that it was not due to my inexperience, I've changed your the 
amount cell of the last line in ~/.../csv2ofx/data/testdefault.csv from 
50000 to 0.50000 and run

csv2ofx defaultModif.csv

only to get

........ValueError: Invalid number format for 0.50000.

The defaultModif.csv file is attached for your convenience.

Again, this could be due to a bug or to my limited knowledge in 
programming, that's why I'm reporting it to you.

I would be more than happy if I were able to find a way to amend 
csv2ofx, but I'm not so smart in programming.

All the best from Italy

command line not running <stdin>??

Traceback (most recent call last):
  File "/usr/local/bin/csv2ofx", line 20, in <module>
    run()
  File "/Users/rob/Downloads/csv2ofx-master/csv2ofx/main.py", line 130, in run
    mtime = p.getmtime(args.source.name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/genericpath.py", line 62, in getmtime
    return os.stat(filename).st_mtime
OSError: [Errno 2] No such file or directory: '<stdin>'

Can't install on Ubuntu 15.04 due to tabutils 0.16.0 requirement

I've just tried to install csv2ofx on a new Ubuntu 15.04 install with the following output:

$ sudo pip install csv2ofx 
Downloading/unpacking csv2ofx
  Downloading csv2ofx-0.18.2-py27-none-any.whl
Downloading/unpacking python-dateutil==2.4.2 (from csv2ofx)
  Downloading python_dateutil-2.4.2-py2.py3-none-any.whl (188kB): 188kB downloaded
Requirement already satisfied (use --upgrade to upgrade): requests==2.4.3 in /usr/lib/python2.7/dist-packages (from csv2ofx)
Downloading/unpacking manage.py==0.2.10 (from csv2ofx)
  Downloading manage.py-0.2.10.tar.gz
  Running setup.py (path:/tmp/pip-build-1Zl2tP/manage.py/setup.py) egg_info for package manage.py

Downloading/unpacking tabutils==0.16.0 (from csv2ofx)
  Could not find a version that satisfies the requirement tabutils==0.16.0 (from csv2ofx) (from versions: 0.9.1, 0.9.4, 0.9.5, 0.10.0, 0.12.0, 0.13.0, 0.18.0, 0.18.1, 0.19.1, 0.19.3, 0.21.0, 0.21.1, 0.22.0, 0.22.1, 0.9.1)
  Some externally hosted files were ignored (use --allow-external to allow).
Cleaning up...
No distributions matching the version for tabutils==0.16.0 (from csv2ofx)
Storing debug log for failure in /home/vagrant/.pip/pip.log

Is this a bug with the script requirements -- specifically tabutils requiring version 0.16.0 which isn't available -- or am I doing something wrong?

csv2ofx not installing

I tried to install csv2ofx on a (pretty vanilla) OSX 10.11.2 system:

~> sudo pip install -v csv2ofx
Downloading/unpacking csv2ofx
  Could not find any downloads that satisfy the requirement csv2ofx
Cleaning up...
  Removing temporary dir /private/tmp/pip_build_root...
No distributions at all found for csv2ofx
Exception information:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/index.py", line 265, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for csv2ofx

Any idea what's wrong?
Thanks.

QFX support?

I have been looking for tools to convert CSV files to QFX files so that Quicken / QuickBooks can import.

Haven't looked at the code for the project.
Can it do it already?

Would you foresee much work modifying it to create QFX files instead?

Thanks.

ofx account_end function drops <LEDGERBAL> section if specified balance is zero

I don't think this is intentional is it?

Line 270 of ofx.py:

if kwargs.get('balance'):

if ofx.footer is called as follows:
ofx.footer(balance=0.0)

then although the kwarg balance exists, it contains a float equal to 0.0, which Python casts to a False

suggest Line 270 is changed to:

if balance in kwargs:

Thanks for your work on this module - very helpful

Paul

Encoding parameter only for reading, not for writing

I think the encoding parameter should only be used to read in files. Anything produced should be in utf-8 regardless.

I have to set the parameter because my bank returns files in windows-1252 but I have zero interest in producing a file in that same format. I think that may be the case for more users.

Alternatively, there could be a new parameter to override the output encoding.

Missing parenthesis syntax error during install

SET PATH=C:\Python367;%PATH%
$ python setup.py install
running install
running build
running build_py
running build_scripts
creating build\scripts-3.6
copying and adjusting csv2ofx -> build\scripts-3.6
running install_lib
creating c:\Python367\Lib
creating c:\Python367\Lib\site-packages
creating c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\csv2ofx.xrc -> c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\csvutils.py -> c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\mappings.py -> c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\ofx.py -> c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\qif.py -> c:\Python367\Lib\site-packages\csv2ofx
copying build\lib\csv2ofx\__init__.py -> c:\Python367\Lib\site-packages\csv2ofx
writing byte-compilation script 'C:\Temp\tmpgz0g804a.py'
c:\Python367\python.exe C:\Temp\tmpgz0g804a.py
  File "c:\Python367\Lib\site-packages\csv2ofx\ofx.py", line 26
    print "Currency not the same."
                                 ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Currency not the same.")?

  File "c:\Python367\Lib\site-packages\csv2ofx\__init__.py", line 71
    print "Using Default Mappings"
                                 ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Using Default Mappings")?

removing C:\Temp\tmpgz0g804a.py
running install_scripts
creating c:\Python367\Scripts
copying build\scripts-3.6\csv2ofx -> c:\Python367\Scripts
running install_egg_info
Writing c:\Python367\Lib\site-packages\csv2ofx-0.2-py3.6.egg-info

Import multiple accounts to MoneyDance fails

Hello All,
Thank you for maintaining this library. The library mapping API, together with the custom mapping support is very welcoming and flexible.

Trying to upload a QIF file with multiple accounts into MoneyDance fails to produce the desired results. The first transaction of each account is associated to the right account while all the others aren't.

I found the "bug" in the generated QIF that is causing it. I don't know if this is a bug in MoneyDance importer or with the generated QIF and I couldn't get that answer from MoneyDance support.

To make it work correctly with MoneyDance the QIF file have the !Type header only for the first transaction of every account

This will not work:

!Account
N4423 (Visa)
TCCard
^
!Type:CCard       <------------------- 1st transaction of the account - this list is mandatory
D01/15/18
Ptransaction 1
T-222.00
^
!Type:CCard         <------------------ not the 1st transaction of the account - this line is problematic
D01/15/18
Ptransaction 2
T-222.00
^

This will work:

!Account
N4423 (Visa)
TCCard
^
!Type:CCard       <------------------- first transaction of the account - this list is mandatory
D01/15/18
Ptransaction 1
T-222.00
^
D01/15/18
Ptransaction 2
T-222.00
^

I forked the library and found it quite easy to fix the "bug". I can create a PR for that but I wanted consult with you about it first to see if this is a something you will merge to the library. I was thinking about extending the mapping the following configuration:

  • Option 1 (focus on the change): 'multiple_accounts_strategy' : 'default' (backward compatible) | 'first_transaction'
  • Option 2 (focus on the target): 'target_app': '' (this is the default) | 'moneydance'

I would appreciate your insights.
Eran.

Help with custom mapping -- missing header keyword: delimiter not working?

I'm trying to convert CVS reports from UBS Switzerland, which come (in French, UTF-8 encoding) like this:

Produit;Monn.;Description;Date de valeur;Description 1;Description 2;Description 3;N° de transaction;Débit;Crédit;Solde
0123 45678900.01C;CHF;Compte personnel UBS;07.11.2017;Ordre e-banking;Payeur 1;Votre facture #123;123456789;334.9;;1000
0123 45678900.01C;CHF;Compte personnel UBS;06.11.2017;Virement postal;Payeur 2;Remboursement;9876543210;;72;737.1  

Starting from the shipped "ubs" mapping, I wrote the following new one (minimal, not complete):

from __future__ import absolute_import
from operator import itemgetter
mapping = {
    'delimiter' : ';',
    'has_header': True, # apparently mandatory...
    'currency'  : itemgetter('Monn.'),
    'account'   : itemgetter('Produit'),
    'amount'    : lambda tr: tr.get('Débit', tr['Crédit']),
    'date'      : itemgetter('Date de valeur'),
}

However (working in a virtualenv based on Python-3.4.6) it doesn't seem to work:

$ csv2ofx -m ubs_ch-fr UBS-CH_export_test.csv 
Field 'Produit' is missing from file. Check `mapping` option.

...with the following stack trace:

Traceback (most recent call last):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/fntools.py", line 629, in byte
    bytes_ = content.encode(ENCODING)
AttributeError: 'itertools.islice' object has no attribute 'encode'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/main.py", line 183, in run
    res = write(dest, IterStringIO(content), **kwargs)
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 1323, in write
    return sum(read_any(filepath, writer, mode, content, **kwargs))
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 445, in read_any
    for line in _read_any(filepath, reader, args, **kwargs):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 371, in _read_any
    for num, line in enumerate(reader(f, *args, **kwargs)):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 1301, in writer
    for chunk in ft.chunk(content, chunksize):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/fntools.py", line 665, in <genexpr>
    generator = (content.read(chunksize) for _ in it.count())
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 138, in read
    return self._read(self.iter, num, False)
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 122, in _read
    byte = ft.byte(content)
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/fntools.py", line 634, in byte
    bytes_ = b''.join(map(encode, content))
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/io.py", line 65, in <genexpr>
    encode = lambda iterable: (s.encode(ENCODING) for s in iterable)
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/ofx.py", line 439, in gen_body
    for datum in data:
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/utils.py", line 152, in gen_data
    for group, main_pos, sorted_trxns in groups:
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/__init__.py", line 294, in clean_trxns
    for grp, trxns in groups:
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/__init__.py", line 280, in gen_trxns
    for grp, transactions in groups:
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/csv2ofx/ofx.py", line 479, in gen_groups
    for gee in group(cleansed, keyfunc):
  File "/home/marcoep/projects/python/_venvs/env1/lib/python3.4/site-packages/meza/process.py", line 588, in group
    sorted_records = sorted(records, key=keyfunc)
KeyError: 'Produit'

What's interesting is that the records list (at meza/process.py:588) looks like

[{'Produit;Monn.;Description;Date de valeur;Description 1;Description 2;Description 3;N° de                
transaction;Débit;Crédit;Solde': '0123 45678900.01C;CHF;Compte personnel UBS;07.11.2017;Ordre              
e-banking;Payeur 1;Votre facture #123;123456789;334.9;;1000'}, {'Produit;Monn.;Description;Date de         
valeur;Description 1;Description 2;Description 3;N° de transaction;Débit;Crédit;Solde': '0123              
45678900.01C;CHF;Compte personnel UBS;06.11.2017;Virement postal;Payeur                                    
2;Remboursement;9876543210;;72;737.1'}]

...as if the delimiter were not taken into account. Any idea?

OFX (v1.x) headers

Thanks for all of the work on this project.

Looking at the OFX output, it appears that the DATA:OFXSGML and ENCODING:UTF-8 headers are added. What do you think of adding other common OFX headers such as:

OFXHEADER:100
VERSION:102
CHARSET:1252
SECURITY:NONE
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE

On Windows GUI launches and CLI doesn't work

On Windows 10 using Python 2.7 (as I could not get Python 3.6.7 to install), launches the GUI only, it does not use any of the command line switches.

After the GUI launches, if I hit the Import button, I get an error.

So at this point, I can't get it to process any of my files.

Looked through main.py and couldn't see anything in there that told it to launch the GUI, or better yet, do not use the GUI.

cd C:\2018
$ where python
c:\Python27\python.exe
c:\Python367\python.exe
C:\Python35\python.exe

$ python -V
Python 2.7.15

$ python C:\csv2ofx_github\csv2ofx-master\csv2ofx -h
Using Default Mappings
Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\csv2ofx\__init__.py", line 151, in OnImport
    style=wx.OPEN|wx.CHANGE_DIR,
AttributeError: 'module' object has no attribute 'OPEN'
GoodBye

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Possible to extract fees from the same line?

CSVs exported from PayPay and Stripe contain two separate columns for the amount: one for the full amount of the transaction, and a second for the fee that Paypal/Stripe collects. Is it possible to use csv2ofx to separate these two amounts so that they can both be accounted for in the OFX?

Pre-processing of CSV

Hello,

Great job on the library it's incredibly useful.

Would it make sense to add a pre-processing sequence the same way you have different mappings ?

My bank has incredibly badly formed CSV so I need to pre-process it to have it match the format that csv2ofx can understand. I am assuming this is something that a lot of people have encountered.

The code would look something like :

from __future__ import absolute_import, print_function

import itertools as it

from meza.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.ofx import OFX
from csv2ofx.mappings.default import mapping

#new import
from csv2ofx.processing.default import pre_process
#new import

ofx = OFX(mapping)

#to be used here
records = read_csv(pre_process('path/to/file.csv'))
#to be used here

groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])

for line in IterStringIO(content):
    print(line)

where pre_process would be a function taking a path and returning a StringIO.
You could then have pre-processing for a bunch of counterparties that do not know how to output proper CSV.
Does any of this make sense ?

Show nice error message when file is missing a mapping field

Currently it crashes

Traceback (most recent call last):
  File "/Users/reubano/Library/Python/2.7/bin/csv2ofx", line 20, in <module>
    run()
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/main.py", line 141, in run
    write(args.dest, IterStringIO(content), **kwargs)
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/io.py", line 691, in write
    return _write(filepath, content, **kwargs)
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/io.py", line 679, in _write
    for c in ft.chunk(content, chunksize):
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/fntools.py", line 248, in <genexpr>
    generator = (content.read(chunksize) for _ in it.count())
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/io.py", line 110, in read
    return self._read(self.iter, n)
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/io.py", line 103, in _read
    return ft.byte(it.islice(iterable, n)) if n else ft.byte(iterable)
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/fntools.py", line 183, in byte
    tupled = tuple(content) if hasattr(content, 'next') else content
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/tabutils/io.py", line 96, in <genexpr>
    return (s.encode(ENCODING) for s in iterable)
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/qif.py", line 233, in gen_body
    for gd in data:
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/utils.py", line 130, in gen_data
    for group, main_pos, sorted_trxns in groups:
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/__init__.py", line 238, in clean_trxns
    for group, trxns in groups:
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/__init__.py", line 224, in gen_trxns
    for group, transactions in groups:
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/qif.py", line 260, in gen_groups
    for group in utils.group_transactions(chnk, keyfunc):
  File "/Users/reubano/Library/Python/2.7/lib/python/site-packages/csv2ofx/utils.py", line 123, in group_transactions
    sorted_transactions = sorted(transactions, key=keyfunc)
KeyError: 'Account'

Exposing QIF account types mapping

Hello,

In qif.py file __init__() function - the account_types is set hard-coded. Not having a way to customize it in the mapping is limiting because and also it is hard to use it since it is hidden.

I will gladly expose it thru the mapping. I think that the current value should be the default if the user did provide one in the mappings. Since the key account_type is already used to map the transaction field for debit/credit, it should be something else.

What do you think?
Eran.

What is the mapping between CSV to OFX trans vs header vs footer

I have experience with several OFX files for bank accounts, but only one example of a csv file from a bank. It would be useful if you could provide some dummy examples for as WYSIWYG instruction.

I understand from the examples I've encountered that an OFX file contains header, transactions, and footer. So I expect the mapping file to do two things:

  • collect the data for the header to construct a minimally valid OFX file
  • reveal the csv column headers which csv2ofx will look for in the csv file export from my bank

First, am I reading the mapping file correctly? The default mapping file is looking for a csv file with the headers, ['Card No.', 'Posted Date', 'Description', 'Credit', 'Debit']? If 'Card No.' refers to the originating bank (my account at my bank), this be an element of the header. So, what is the mapping between CSV headers (optional, mandatory) to which OFX parts (transactions, header, footer).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.