fruitl00p / php-mt940 Goto Github PK

A mt940 parser in PHP

License: MIT License

PHP 100.00%

parser php

php-mt940's Introduction

php-mt940?

The php-mt940 package provides a lightweight parser for MT940 format which is used by SWIFT. The output is transformed into easy to use dataclasses Transaction_banking which itself contains Statement_banking objects. Pretty straight forward.

Requirements

At least PHP 7

Installation

If composer is not yet on your system, follow the instructions on getcomposer.org to do so.

To add this dependency to your project, simply run the following command from the root of your project:

composer require kingsquare/php-mt940

This ensures that you install the latest stable release.

How to use the parser?

I've attached a simple script in the examples directory to explain it a bit more in detail, but after loading the required classes, the usage should be pretty simple:

<?php
// ... load everything ... //

// instantiate the actual parser
// and parse them from a given file, this could be any file or a posted string
$parser = new \Kingsquare\Parser\Banking\Mt940();
$tmpFile = __DIR__.'/test.mta';
$parsedStatements = $parser->parse(file_get_contents($tmpFile));
?>

Included engines

Currently the following engines are included:

ABNAMRO (here)
ING (here)
KNAB (here)
RABOBANK (here)
SPARKASSE (here)
TRIODOS (here)
HSBC (here)
SNS (here) Experimental
BUNQ (here)
PENTA (here)
ASN (here)
KBS (here)
ZETB (here)
KONTIST (here)
a default UNKNOWN-engine (here)

Custom engines

To override engines or just try a one-off engine on a file, you can pass an engine into the parse-method:

<?php
// ... load everything ... //

class MyCustomMt940Engine extends \Kingsquare\Parser\Banking\Mt940\Engine {
    // add your overrides / overloads and custom logic here
}

// instantiate the actual parser
// and parse them from a given file, this could be any file or a posted string
$parser = new \Kingsquare\Parser\Banking\Mt940();
$engine = new MyCustomMt940Engine();
$tmpFile = __DIR__.'/test.mta';
$parsedStatements = $parser->parse(file_get_contents($tmpFile), $engine);
?>

Known issues

I've provided unittests but please take a look at the github issue tracker for the latest ideas's, issues or other stuff..

Future plans

I do intend to add new engines or keep everything running smoothly, but since i don't have access to any more test files, it's hard to add new engines ;) The unknown engine should work or atleast give some idea as to where different banks diverge from the standard. If you do have any ideas, examples or new banks that you'd like to see incorporated, please don't hesitate and send me an issue / pullrequest!

Contact

This is GitHub, you know where to find me :)

License

PHP-MT940 is licensed under the MIT License - see the LICENSE file for details

php-mt940's People

Contributors

Stargazers

Watchers

Forkers

markoheijnen tuxphr34k bravoman tpokorra ixolit sven-codeculture codeculturehq maennchen hpolthof dfcplc beechit accountingbox delassiter marksdestiny martijnengler doenietzomoeilijk jicked yapsr tomcoonen ahmad-moussawi pverdu atanenl pbxg33k robquistnl berlusgmbh mathieumaas mnolte kantoor-f12 ahoicloud printdeal dwd-build sevannerse nssnl ddcatgg patrickvanharen sebastianberm casaicn groupnet punktdeforks vladshut uwem backendtea gertjantervrugt mic1983 informeronline paulvankaam peterjumpnl lechu4444 retaildesk imrerose avramflorin sinanbozca dembyp sharkooon rud5g ttibensky mrzegocki kefkef123 tactiplan cuneytsenturk

php-mt940's Issues

csv - parcer

Hello, would it also be able to create a parcer that will be able to convert .csv files into a digital bankstatement.

Only business bank accounts have the option to generate a mt940-file. People who don't have business account can't use this parcer. It would be great if this option could be included in the current parcer. Or maybe a second parcer can be created to convert .csv files into digital bank statements.

I have added a manual from the Rabobank. The descripe how the file is build. I can provide more examples of .csv files from different banks if needed. But I want to do this via personal email. So please let me know when you need information.

Thanks,
Boris

Registering new Engines doesn't respect priority

You can register your own Engine with Mt940\Engine::registerEngine(MyCustomRaboEngine::class, 1000); but the auto detect doesn't respect the priority.

A sorting of self::$registeredEngines by key should be added in Engine::detectBank() to respect the priority

Correction to ABNAMRO description + AccountName filtering

We have to process ABNAMRO statements and the accountName and Description are not processed as excpected. Even the iban is not found for "SEPA INCASSO BEDRIJVEN DOORLOPEND".

Input:
:61:1907190719C123,4N654NONREF
:86:/TRTP/SEPA OVERBOEKING/IBAN/NL07RABO0xxxxxxxxx/BIC/RABONL2U/NAME/
nnnnnnnnnnnnnnnnnn/REMI/oooooooooooooooooooooooooooooooo/EREF/NOTPR
OVIDED

results into object:
[account:Kingsquare\Banking\Transaction:private] => NL07RABO0xxxxxxxxx
[accountName:Kingsquare\Banking\Transaction:private] => nnnnnnnnnnnnnnnnn/REMI/oooooooooooooooooooooooooooooooo/EREF/NOTPR
[price:Kingsquare\Banking\Transaction:private] => 123.4
[debitcredit:Kingsquare\Banking\Transaction:private] => C
[cancellation:Kingsquare\Banking\Transaction:private] =>
[description:Kingsquare\Banking\Transaction:private] => /TRTP/SEPA OVERBOEKING/IBAN/NL07RABO0xxxxxxxxx/BIC/RABONL2U/NAME/nnnnnnnnnnnnnnnnn/REMI/oooooooooooooooooooooooooooooooo/EREF/NOTPROVIDED
[valueTimestamp:Kingsquare\Banking\Transaction:private] => 1563487200
[entryTimestamp:Kingsquare\Banking\Transaction:private] => 1563487200
[transactionCode:Kingsquare\Banking\Transaction:private] => 654:

Improved abn.php functions adding extra preg_match lines:

protected function parseTransactionAccountName()
{
$results = parent::parseTransactionAccountName();
if ($results !== '') {
return $results;
}
$results = [];
$data = $this->getCurrentTransactionData();

    // SEPA MT940 Structured
    if (preg_match('#/NAME/(.+?)\n?/(REMI|IBAN|BIC|ADDR|ISDT|CSID|MARF)/#ms', $data, $results)) {
        $accountName = trim($results[1]);
        if (!empty($accountName)) {
            return $this->sanitizeAccountName($accountName);
        }
    }

    if (preg_match('/:86:(GIRO|BGC\.)\s+[\d]+ (.+)/', $data, $results)
        && !empty($results[2])
    ) {
        return $this->sanitizeAccountName($results[2]);
    }

    if (preg_match('/:86:.+\n(.*)\n/', $data, $results)
        && !empty($results[1])
    ) {
        return $this->sanitizeAccountName($results[1]);
    }

    return '';
}

protected function parseTransactionDescription()
{
    $results = [];
    $data = $this->getCurrentTransactionData();

    if (preg_match('#/REMI/(.+?)\n?/(EREF|IBAN|BIC|ADDR|ISDT|CSID|MARF)/#ms', $data, $results)) {
        $description = trim($results[1]);
        if (!empty($description)) {
            return $this->sanitizeDescription($description);
        }
    }

    if (preg_match('/:86:.+\n(.*)\n/', $data, $results)
        && !empty($results[1])
    ) {
        return $this->sanitizeDescription($results[1]);
    }
    return '';
}

Now resulting into:
[account:Kingsquare\Banking\Transaction:private] => NL07RABO0xxxxxxxxx
[accountName:Kingsquare\Banking\Transaction:private] => nnnnnnnnnnnnnnnnn
[price:Kingsquare\Banking\Transaction:private] => 123.4
[debitcredit:Kingsquare\Banking\Transaction:private] => C
[cancellation:Kingsquare\Banking\Transaction:private] =>
[description:Kingsquare\Banking\Transaction:private] => oooooooooooooooooooooooooooooooo
[valueTimestamp:Kingsquare\Banking\Transaction:private] => 1563487200
[entryTimestamp:Kingsquare\Banking\Transaction:private] => 1563487200
[transactionCode:Kingsquare\Banking\Transaction:private] => 654

File and folder case

All the src files and folders are lowercase. This might work on Mac and Windows where the fs is case-insensitive, but it doesn't work in Linux.

ABNAMRO Transaction Entry Date parsing bug (using current years instead of booking year)

Typical bug cases; we just had a customer that was reading in bank transactions, and the dates were 2017-12-15 instead of 2016-12-15.

The cause was found here; https://github.com/fruitl00p/php-mt940/blob/master/src/Parser/Banking/Mt940/Engine/Abn.php#L87-L94

As you can see in this code, only a month and date seem to be read in. The PHP Date function gets fed "mmdd" instead of "yymmdd" - and it works fine for the biggest part of the year. It will only fail if you're reading in transactions from a different year.

We had some transactions on that were 16-12-15 but got read in as 17-12-15 because the date function got fed "1215".

An example transaction looks like this:

ABNANL2A
940
ABNANL2A
:20:ABN AMRO BANK NV
:25:534334164
:28:35001/1
:60F:C161214EUR1124,58
:61:1612151215C109,65N654NONREF
:86:/TRTP/SEPA OVERBOEKING/IBAN/NL75INGB0001234567/BIC/INGBNL2A/NAME/
HR K XXXXXXX EN/OF MW H XXXXXXX/REMI/RES.NR 00042/EREF/NOTPROV
IDED
:61:1612151215D62,63N247NONREF
:86:/TRTP/SEPA INCASSO ALGEMEEN DOORLOPEND/CSID/NL36ZZZ123456780000
/NAME/XXXXXXXX B.V./MARF/000000059990/REMI//INV/1511682351 1.1
2.2016/IBAN/NL37DEUT0123456789/BIC/DEUTNL2AXXX/EREF/0123456789123
61000
:62F:C161215EUR1171,6
-

As you can see on this line there's 2 dates - 1 with a year and one without;

:61:161215---1215C109,65N654NONREF

We need to take the year in account or bank imports overlapping different years will break.

I'll be opening a PR for this issue.

Basic example

A basic example would come in handy.

I've already been searching for an hour to how exactly use this class.

How do you iterate over the parsedStatement variable?
What functions are available to get information from found transactions?

Why is this nowhere documented?

ING sanitizeDescription update

The description of the ING transactions hold more information which is discarded by this parser. For example Direct Debet (automatische incasso) transactions have their desscription next to the PREF identifier. Below is a function that adds this information to the description. Feel free to implement/rewrite this or do nothing with it ;)

    /**
     * Overloaded: ING encapsulates the description with /REMI/ for SEPA.
     *
     * {@inheritdoc}
     */
    protected function sanitizeDescription($string)
    {
        $description = parent::sanitizeDescription($string);
        if (strpos($description, '/PREF/') !== false
                && preg_match('#/PREF/(.*?)/#s', $description, $results) && !empty($results[1])
        ) {
            $description_start = $results[1];
        }
        if (strpos($description, '/EREF/') !== false
                && preg_match('#/EREF/(.*?)/#s', $description, $results) && !empty($results[1])
        ) {
            $description_start = $results[1];
        }
        if (strpos($description, '/REMI/USTD//') !== false
                && preg_match('#/REMI/USTD//(.*?)/$#s', $description, $results) && !empty($results[1])
        ) {
            $description_end = $results[1];
        }
        if (strpos($description, '/REMI/STRD/CUR/') !== false
                && preg_match('#/REMI/STRD/CUR/(.*?)/#s', $description, $results) && !empty($results[1])
        ) {
            $description_end = $results[1];
        }

        return $description_start.' '.$description_end;
    }

No result and no errors?

Hello,

I'm testing this parser on a sta file from ING (Poland) using this code:

<?php
require 'vendor/autoload.php';
$mt940file = __DIR__ . '/test.sta';
$parser = new \Kingsquare\Parser\Banking\Mt940();
$engine = new \Kingsquare\Parser\Banking\Mt940\Engine\Ing();
$parsedStatements = $parser->parse(file_get_contents($mt940file), $engine);
var_dump($parsedStatements);

but all I get is:

array(0) {
}

Am I missing something? Or maybe is my file format not supported?

price is not correctly parsed

Example line from MT940 file
:61:1807300730D28,5N132000002018827922//B8G30PGA01UD901N

Correct price: Debit 28,50
Parsed price: Debit 901,00

Problem comes from the regex in parseTransactionPrice in the Engine.php
The pattern /^:61:.*[CD]([\d,\.]+)N/i has a .* in it which is greedy, when you add a ? after the .* it becomes non greedy and matches the first [CD]([\d,\.]+)N in the line.

Correct regex: /^:61:.*?[CD]([\d,\.]+)N/i

Add code descriptions for Rabobank.

The file is here:
https://www.rabotransact.com/rtpubeu/ContentIntegrationPortlet/images/Transaction_description_RCM-RDC_v2.6NL_tcm79-187434.pdf

Writing MT940

Hi, can ths package also be used to create and write MT940 files? If so, how?

Classifying transactions.

There are several transaction types that can be identified for all accounts regardless of bank.
It would be good to add that type to make handling of the transaction easier.
(Currently I would have to parse the bank specific transaction code, which I do not want to know about).

Currently I have identified the following types:

    const SEPA_TRANSFER = 10;
    const SEPA_DIRECTDEBIT = 20;
    const BANK_COSTS = 30;
    const BANK_INTEREST = 31;
    const ATM_WITHDRAWAL = 40;
    const PAYMENT_TERMINAL = 50;
    const UNKNOWN = 99;

These are already in my fork (https://github.com/SamMousa/php-mt940/tree/feature-interfaces-2), however I have made some design decisions that you might not agree with / need to be discussed.

I have included a ValueObjects library.
I have added more to the interfaces.

If you are interested in adopting these changes, it might be wise to create a new develop branch.

I will be actively developing this library regardless, the downside is I will probably not have time to upgrade all parsers or properly update all tests.

Let me know how / if you think we could best work together!

Transaction::getAccountName return type

Transaction::getAccountName() should return null when no name is present in the transaction; returning an empty string is not the same.

update the ABN engine to 1.4

According to the docs several fields have been updated for the new SWIFT + SEPA specs. Some additional tests are needed too

decide on handling BC issues (keep old or dump it?)
update the engine
add tests to confirm the 1.4 spec

getRelativePrice() missing

AFter installation using composer as instructed the getRelativePrice function inside Transaction.php is missing.

Composer seems to be installing a different version as is shown on github?
Inside my composer.json: "kingsquare/php-mt940": "^1.4"

Also, why are we on fruitl00p instead of Kingsquare?

Separate factory from base engine.

Currently Kingsquare\Parser\Banking\Mt940\Engine is both a parser and a factory.

It would be cleaner to create a separate ParserFactory and separate the logic of registering / finding / creating parsers from the parsers themselves.

Rabobank /PREF handling

A small improvement could be made by parsing a description from:
/PREF/This is the payment description
to:
This is the payment description

Knab transactions

I am testing this package out by importing my Knab Mt940 statements.
All incoming funds (customers paying their invoice) are being marked as Credit ('C'), while my own payments, such as my phone bill, are being marked as Debit ('D').
Should this not be the other way around?

startPriceDate, endPriceDate?

Hey fruitl00p, I was wondering if you could add the possibility to get a startPriceDate and an endPriceDate out of a mt940-file to your library? Thanks in advance!

[QUESTION] Is this still being maintained?

Hi! Is this project still being maintained? The last commit is almost 1 year old.

This library runs on php8, but it produces warnings because of missing return types
Return type of Kingsquare\Banking\Transaction::jsonSerialize() should either be compatible with JsonSerializable::jsonSerialize(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice

ING EntryDate fix

I've used this package for years without any problems at all, but suddenly in the past months I got a few transactions with wrong dates. The wrong dates occurred with bounced direct debit transactions. Those transactions are given (by ING) a "value date" which sometimes is weeks earlier than the "entry date".

The main parser doesn't parse the entry date. It only uses the value date and copies the date to the entry date.

According to the spec the Value/Entry format is:
YYMMDD[MMDD]
which is the same as used in the ABN engine.

So I copied the parseTransactionEntryTimestamp function from ABN to ING, and the date parsing is fixed now.

Anyways, thanks for the great package!

Statements in different currencies to not have correct Start/End price

The regular expression used to parse the statements has a hardcoded EUR, where the currency could be something else. This results in an empty start and end balance value.

The regex:
/:62F:([CD])?.*EUR([\d,\.]+)*/

could be:
/:62F:([CD])?.*(?:[A-Z]{3})([\d,\.]+)*/

But then there need to be testfiles with different currencies also, and perhaps it would be nice to actually capture the currency and add it to the statement.

Can't parse file

My file begins from:
STARTDISP
Why it unparsable by 2 days?

Problem parsing ABN Amro mt940 'Dagafschrift'

Hi,

We're making an application where we read an ABN Amro mt940 file that contains multiple statements. Problem is that this file doesn't contain the :20: tag and therefor there aren't any statements when reading this type of file.
The file is downloaded straight from the ABN Amro website and looks like this:

ABNANL2A
940
ABNANL2A
:25:NL62ABNAXXXXXXX
:28:XXX/XX
:60F:C161111EURXXXXXX,XX
:61:1611141114CXXX,XXNXXXXXXXX
:86:/TRTP/SEPA OVERBOEKING/IBAN/NL37ABNAXXXXXXXX/BIC/ABNANL2A/NAME/
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:61:1611141114CXXX,XXNXXXXXXXX
:86:/TRTP/SEPA OVERBOEKING/IBAN/NL54ABNAXXXXXXXX/BIC/ABNANL2A/NAME/
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:61:1611141114CXXXX,XXNXXXXXXXX
:86:/TRTP/SEPA OVERBOEKING/IBAN/NL28ABNAXXXXXXXX/BIC/ABNANL2A/NAME/
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:62F:C161114EURXXXXXX,XX
:64:C161114EURXXXXXX,XX
:65:C161115EURXXXXXX,XX
:65:C161116EURXXXXXX,XX
:65:C161117EURXXXXXX,XX
:65:C161118EURXXXXXX,XX
:65:C161119EUEXXXXXX,XX
:65:C161120EURXXXXXX,XX
-

The example above is repeated a couple of times in 1 file.
Do you have any information about this type of mt940 files or do you possibly have a solution so it's possible to read this type of mt940 file?

Thanks in advance

Can you modify your code

I will use your programm in a symfony2 project but I have a unknown mt940 file and symfony stops every time by "trigger_error". My file starts at first line with :20: folowed by date of file formated like "ymd".
Please delete line 68 in Engine.php and fill Unknown.php whit folowing code:

getCurrentStatementData(), $results) && !empty($results[1]) ) { return $this->sanitizeAccount($results[1]); } // SEPA / IBAN if (preg_match('/:25:([A-Z0-9]{8}[\d.]+)*/', $this->getCurrentStatementData(), $results) && !empty($results[1]) ) { return $this->sanitizeAccount($results[1]); } return ''; } /** - Overloaded: Sparkasse uses 62M and 62F - @inheritdoc _/ protected function parseStatementEndPrice() { $results = []; if (preg_match('/:62[FM]:._EUR([\d,.]+)*/', $this->getCurrentStatementData(), $results) && !empty($results[1]) ) { return $this->sanitizePrice($results[1]); } return ''; } /** - Overloaded: Sparkasse uses 60M and 60F - @inheritdoc _/ protected function parseStatementStartPrice() { $results = []; if (preg_match('/:60[FM]:._EUR([\d,.]+)*/', $this->getCurrentStatementData(), $results) && !empty($results[1]) ) { return $this->sanitizePrice($results[1]); } return ''; } }

KNAB parser fixes

@SamMousa I've received a private test file from KNAB which gives some problems with the current KNAB parser...

I've altered some of the parsing. Since KNAB is still working on structured MT940 files and they've requested i keep the test file to myself, could you take a look at the pull request ( #32 )

Thanks!

PHP version requirement

I'm thinking of dropping 5.3 support to urge the few users of the package to reconsider using 5.3 today in production :)

Thus I'm gauging the audience into response. The idea is to drop 5.3 (and possibly 5.4) before php-mt940 1.0

any thoughts?

KNAB - New structured file

KNAB now ahs the ability to export to the old format and a new structured one.
The next transaction is an example of the new format:

:61:1604290429D96,84N105NONREF//B6D29PGWUM0RDZ4H :86:/TRTP/SEPA ACCEPTGIRO/IBAN/NL83ABNA0437015300/BIC/ABNANL2AXXX /NAME/ACHMEA SCHADEVERZEKERINGEN N.V./EREF/5002100023310160 /REMI/EERSTE MAAND, MAART VERZEKERING 5002100023 310160

It's not a big change (i think)... But are you able te create a fix for this?

Add interfaces

There should be an interface for parsers instead of a base object.
The base parser could then implement the interface (partially or fully).
That would allow external parsers to implement the interface in their own way.