Comments (6)
You will need to have the raw input and the reference transcription aligned like in the ICDAR 2019 competition. Figure 5 in the paper describes how to train the sequence-to-sequence model.
Look at the notebooks folder to see the data pipeline for each of the languages.
You may also be interested in the updated version of the library I used to train the sequence models.
The code below should do the trick to align the input and the reference
from nltk.metrics.distance import edit_distance_align as d
def align(s1, s2, fill = "@"):
mapping = d(s1, s2, substitution_cost = 2)
o1, o2 = "", ""
for i, (a, b) in enumerate(mapping[1:], 1):
if mapping[i - 1][0] == a or i - 1 < 0 :
o1 += fill
else:
o1 += s1[a - 1]
if mapping[i - 1][1] == b or i - 1 < 0 :
o2 += fill
else:
o2 += s2[b - 1]
print(o1)
print(o2)
return o1, o2
# sample input
align("INEVR jrlf Major Long ow.", "I NEVER Major Longhow.")
# sample output
('I@NEV@R jrlf Major Long ow.', 'I NEVER @@@@@Major Longhow.')
from post_ocr_correction.
Thank for your help . It's can train with word not sentences
from post_ocr_correction.
Maybe you can translate that last sentence with Google Translate? You can train the sequence-to-sequence models with words, but it's best if you train them with n-grams, like in the paper.
from post_ocr_correction.
Thank a lot i will try
from post_ocr_correction.
hi bác @ThorPham
Không biết bác đã train được chưa nhỉ, nếu được thì bác cho em xin checkpoint của bác để thử nghiệm được không ạ
Em cảm ơn
from post_ocr_correction.
Please translate?
from post_ocr_correction.
Related Issues (9)
- Will you able to share the pretrained model? HOT 1
- vocabulary not found HOT 1
- Dealing with long text HOT 2
- ModuleNotFoundError: No module named 'pytorch_decoding' HOT 1
- evaluate_model HOT 3
- Discussion HOT 8
- How can I train the model with the dataset in the paper? HOT 4
- For the inference, how can we do the alignment? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from post_ocr_correction.