How to train with other language ? about post_ocr_correction HOT 4 CLOSED

You will need to have the raw input and the reference transcription aligned like in the ICDAR 2019 competition. Figure 5 in the paper describes how to train the sequence-to-sequence model.

Look at the notebooks folder to see the data pipeline for each of the languages.

You may also be interested in the updated version of the library I used to train the sequence models.

The code below should do the trick to align the input and the reference

from nltk.metrics.distance import edit_distance_align as d

def align(s1, s2, fill = "@"):
    mapping = d(s1, s2, substitution_cost = 2)
    o1, o2 = "", ""
    for i, (a, b) in enumerate(mapping[1:], 1):
        if mapping[i - 1][0] == a or i - 1 < 0 :
            o1 += fill
        else:
            o1 += s1[a - 1]
        
        if mapping[i - 1][1] == b or i - 1 < 0 :
            o2 += fill
        else:
            o2 += s2[b - 1]
    print(o1)
    print(o2)
    return o1, o2

# sample input
align("INEVR jrlf Major Long ow.", "I NEVER Major Longhow.")

# sample output
('I@NEV@R jrlf Major Long ow.', 'I NEVER @@@@@Major Longhow.')

from post_ocr_correction.

Thank for your help . It's can train with word not sentences

from post_ocr_correction.

Maybe you can translate that last sentence with Google Translate? You can train the sequence-to-sequence models with words, but it's best if you train them with n-grams, like in the paper.

from post_ocr_correction.

Thank a lot i will try

from post_ocr_correction.