Comments (32)
@Hrishkesh , @sathvikask0, @Karthik1904
Guys, I have written a simple file to run Tesseract ocr and output a json file in the format as in invoice_data/ example:
https://github.com/hhien/tesseract_applications.git
from cutie.
Hi,
Can you please provide details on how to generate and test the model? There are a whole lot of files and command line arguments to be given. Can you please update README as to how this model has to be trained and tested?
from cutie.
@Ibmaria
Hello 4kssoft,
Please how do you get this format ?.I have no idea .Can you explain me please?
Thanks in advanceTo get the format. I analyzed this file https://github.com/vsymbol/CUTIE/blob/master/data_loader_json.py
I'm training a model with these parameters:
python main_train_json.py \ --doc_path 'invoice_data/' \ --save_prefix 'INVOICE' \ --test_path '' \ --embedding_file '' \ --ckpt_path 'graph/' \ --ckpt_file 'CUTIE_highresolution_8x_d20000c9(r80c80_iter_40000.ckpt' \ --tokenize True \ --update_dict True \ --dict_path 'dict/' \ --rows_segment 72 \ --cols_segment 72 \ --augment_strategy 1 \ --positional_mapping_strategy 1 \ --rows_target 64 \ --cols_target 64 \ --rows_ulimit 80 \ --fill_bbox False \ --data_augmentation_extra True \ --data_augmentation_dropout 1 \ --data_augmentation_extra_rows 16 \ --data_augmentation_extra_cols 16 \ --batch_size 32 \ --iterations 40000 \ --lr_decay_step 13000 \ --learning_rate 0.0001 \ --lr_decay_factor 0.1 \ --hard_negative_ratio 3 \ --use_ghm 0 \ --ghm_bins 30 \ --ghm_momentum 0 \ --log_path 'log/' \ --log_disp_step 100 \ --log_save_step 100 \ --validation_step 100 \ --test_step 400 \ --ckpt_save_step 50 \ --embedding_size 128 \ --weight_decay 0.0005 \ --eps 1e-6
Could you please provide the ckpt file CUTIE_highresolution_8x_d20000c9(r80c80_iter_40000.ckpt
and the invoice dataset that you had used for training?
from cutie.
Hello @vsymbol
Please how do you get this format ?.I have no idea .Can you explain me please?
Thanks in advance
from cutie.
Hello @vsymbol , http://52.193.30.103 seems to be down. Could you provide the updated link?
from cutie.
@4kssoft
Thanks a lot for sharing how to train the model . however how engine(api) did you use to get the boxes coordinates from the images ?
Thanks
I use own software for labeling documents (https://www.youtube.com/watch?v=1okRMNxC0ec)
from cutie.
from cutie.
Hello,
I couldn't find the exact input data to train the model on the icdar dataset , can you provide explanations for it ?
thanks.
from cutie.
Hello,
Could you provide your input data for the model to reproduce the results or at least the input data format so that I can try the model on my custom dataset
The project is refreshed with all history removed. All programs are runnable expect that the data example is not uploaded.
You may infer the correct data format from the data_loader_json.py file. Pull request is welcomed for making the project runnable out of the box. I'll let you know when the original data format can be provided, otherwise please feel free to create a pull request.
from cutie.
Please tell me if this is the correct data format?
Format:
file_name.json
{
"global_attributes": {
"file_id": "$file_name"
},
"fields":[
{
"field_name": "$class_name",
"key_id": [],
"key_text": [],
"value_id": [$word_id],
"value_text": "$word_text"
},...
],
"text_boxes":[
{
"id": $word_id,
"bbox": [$word_x_min, $word_y_min, $word_x_max, $word_y_max],
"text": "$word_text"
},...
]
}
Example:
file1.json
{
"global_attributes": {
"file_id": "file1.jpg"
},
"fields":[
{
"field_name": "class1",
"key_id": [],
"key_text": [],
"value_id": [1],
"value_text": "sample1"
},
{
"field_name": "class1",
"key_id": [],
"key_text": [],
"value_id": [2],
"value_text": "sample2"
},
{
"field_name": "class2",
"key_id": [],
"key_text": [],
"value_id": [3],
"value_text": "sample3"
}
],
"text_boxes":[
{
"id": 1,
"bbox": [10, 10, 50, 20],
"text": "sample1"
},
{
"id": 2,
"bbox": [55, 10, 100, 20],
"text": "sample2"
},
{
"id": 3,
"bbox": [50, 30, 100, 40],
"text": "sample3"
}
]
}
Or maybe the correct format should look like this
Example 2:
file1.json
{
"global_attributes": {
"file_id": "file1.jpg"
},
"fields":[
{
"field_name": "class1",
"key_id": [],
"key_text": [],
"value_id": [1, 2],
"value_text": ["sample1", "sample2"]
},
{
"field_name": "class2",
"key_id": [],
"key_text": [],
"value_id": [3],
"value_text": ["sample3"]
}
],
"text_boxes":[
{
"id": 1,
"bbox": [10, 10, 50, 20],
"text": "sample1"
},
{
"id": 2,
"bbox": [55, 10, 100, 20],
"text": "sample2"
},
{
"id": 3,
"bbox": [50, 30, 100, 40],
"text": "sample3"
}
]
}
from cutie.
Hello @vsymbol , http://52.193.30.103 seems to be down. Could you provide the updated link?
I too am unable to reach 52.193.30.103 even via ping. Can you confirm if this is up?
from cutie.
Sample data file #8 (comment)
from cutie.
Hello @vsymbol , http://52.193.30.103 seems to be down. Could you provide the updated link?
I too am unable to reach 52.193.30.103 even via ping. Can you confirm if this is up?
Hi varshaneya, the link is down.
from cutie.
@Ibmaria
Hello 4kssoft,
Please how do you get this format ?.I have no idea .Can you explain me please?
Thanks in advance
To get the format. I analyzed this file https://github.com/vsymbol/CUTIE/blob/master/data_loader_json.py
I'm training a model with these parameters:
python main_train_json.py \ --doc_path 'invoice_data/' \ --save_prefix 'INVOICE' \ --test_path '' \ --embedding_file '' \ --ckpt_path 'graph/' \ --ckpt_file 'CUTIE_highresolution_8x_d20000c9(r80c80_iter_40000.ckpt' \ --tokenize True \ --update_dict True \ --dict_path 'dict/' \ --rows_segment 72 \ --cols_segment 72 \ --augment_strategy 1 \ --positional_mapping_strategy 1 \ --rows_target 64 \ --cols_target 64 \ --rows_ulimit 80 \ --fill_bbox False \ --data_augmentation_extra True \ --data_augmentation_dropout 1 \ --data_augmentation_extra_rows 16 \ --data_augmentation_extra_cols 16 \ --batch_size 32 \ --iterations 40000 \ --lr_decay_step 13000 \ --learning_rate 0.0001 \ --lr_decay_factor 0.1 \ --hard_negative_ratio 3 \ --use_ghm 0 \ --ghm_bins 30 \ --ghm_momentum 0 \ --log_path 'log/' \ --log_disp_step 100 \ --log_save_step 100 \ --validation_step 100 \ --test_step 400 \ --ckpt_save_step 50 \ --embedding_size 128 \ --weight_decay 0.0005 \ --eps 1e-6
from cutie.
@4kssoft
Thanks a lot for sharing how to train the model . however how engine(api) did you use to get the boxes coordinates from the images ?
Thanks
from cutie.
@4kssoft
Thanks you !
from cutie.
@4kssoft
Thanks a lot for sharing how to train the model . however how engine(api) did you use to get the boxes coordinates from the images ?
ThanksI use own software for labeling documents (https://www.youtube.com/watch?v=1okRMNxC0ec)
@4kssoft Thanks for sharing. How can i access you tool ?
from cutie.
@4kssoft
Thanks a lot for sharing how to train the model . however how engine(api) did you use to get the boxes coordinates from the images ?
ThanksI use own software for labeling documents (https://www.youtube.com/watch?v=1okRMNxC0ec)
@4kssoft Thanks for sharing. How can i access you tool ?
This is a beta version for now. I plan to publish this software, but not as open source
from cutie.
@Ibmaria
Hello 4kssoft,
Please how do you get this format ?.I have no idea .Can you explain me please?
Thanks in advanceTo get the format. I analyzed this file https://github.com/vsymbol/CUTIE/blob/master/data_loader_json.py
I'm training a model with these parameters:
python main_train_json.py \ --doc_path 'invoice_data/' \ --save_prefix 'INVOICE' \ --test_path '' \ --embedding_file '' \ --ckpt_path 'graph/' \ --ckpt_file 'CUTIE_highresolution_8x_d20000c9(r80c80_iter_40000.ckpt' \ --tokenize True \ --update_dict True \ --dict_path 'dict/' \ --rows_segment 72 \ --cols_segment 72 \ --augment_strategy 1 \ --positional_mapping_strategy 1 \ --rows_target 64 \ --cols_target 64 \ --rows_ulimit 80 \ --fill_bbox False \ --data_augmentation_extra True \ --data_augmentation_dropout 1 \ --data_augmentation_extra_rows 16 \ --data_augmentation_extra_cols 16 \ --batch_size 32 \ --iterations 40000 \ --lr_decay_step 13000 \ --learning_rate 0.0001 \ --lr_decay_factor 0.1 \ --hard_negative_ratio 3 \ --use_ghm 0 \ --ghm_bins 30 \ --ghm_momentum 0 \ --log_path 'log/' \ --log_disp_step 100 \ --log_save_step 100 \ --validation_step 100 \ --test_step 400 \ --ckpt_save_step 50 \ --embedding_size 128 \ --weight_decay 0.0005 \ --eps 1e-6
@4kssoft if possible please provide the pretrained model that you are using!
And guys for the annotation with bounding boxes please look into this link, might be useful :
Tesseract OCR: Text localization and detection
from cutie.
@4kssoft Hi I have my own data and extracted text using OCR tesseract and got the position of each word, can i know how to get in the format you showed an example in your repository for sample pdf file Faktura1.pdf_0.json how to get in this format and i need in the format you done can you let me know ???
from cutie.
@4kssoft Thanks for your suggestions, I have generated my own training datasets and i am able to train the model, but I am not getting what should be the input format to predict the result. If you know what modification it requires to get the result please just inform us.
from cutie.
Hello all
@4kssoft if possible please provide the pretrained model that you are using!
@sathvikask0
Sorry but unfortunately I cannot share my model
@4kssoft Hi I have my own data and extracted text using OCR tesseract and got the position of each word, can i know how to > get in the format you showed an example in your repository for sample pdf file Faktura1.pdf_0.json how to get in this format > > and i need in the format you done can you let me know ???
@Hrishkesh
as I wrote #7 (comment) earlier, I use my own tool to annotate documents, I also have ready functions for exporting training data to various models.
I'm planning to publish a beta version of my solution soon
@4kssoft Thanks for your suggestions, I have generated my own training datasets and i am able to train the model, but I am >not getting what should be the input format to predict the result. If you know what modification it requires to get the result ?>please just inform us.
@Neelesh1121
The format is the same as for training. Look at the https://github.com/vsymbol/CUTIE/blob/master/main_evaluate_json.py script
from cutie.
everytime i try to use main_evaluate_json.py I get this error
@4kssoft @samhita-alla @vsymbol
Can anyone please help
2 root error(s) found.
(0) Not found: Key feature_fuser/biases not found in checkpoint
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
(1) Not found: Key feature_fuser/biases not found in checkpoint
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[save/RestoreV2/_49]]
0 successful operations.
0 derived errors ignored.
from cutie.
@Ibmaria
Hello 4kssoft,
Please how do you get this format ?.I have no idea .Can you explain me please?
Thanks in advanceTo get the format. I analyzed this file https://github.com/vsymbol/CUTIE/blob/master/data_loader_json.py
I'm training a model with these parameters:
python main_train_json.py \ --doc_path 'invoice_data/' \ --save_prefix 'INVOICE' \ --test_path '' \ --embedding_file '' \ --ckpt_path 'graph/' \ --ckpt_file 'CUTIE_highresolution_8x_d20000c9(r80c80_iter_40000.ckpt' \ --tokenize True \ --update_dict True \ --dict_path 'dict/' \ --rows_segment 72 \ --cols_segment 72 \ --augment_strategy 1 \ --positional_mapping_strategy 1 \ --rows_target 64 \ --cols_target 64 \ --rows_ulimit 80 \ --fill_bbox False \ --data_augmentation_extra True \ --data_augmentation_dropout 1 \ --data_augmentation_extra_rows 16 \ --data_augmentation_extra_cols 16 \ --batch_size 32 \ --iterations 40000 \ --lr_decay_step 13000 \ --learning_rate 0.0001 \ --lr_decay_factor 0.1 \ --hard_negative_ratio 3 \ --use_ghm 0 \ --ghm_bins 30 \ --ghm_momentum 0 \ --log_path 'log/' \ --log_disp_step 100 \ --log_save_step 100 \ --validation_step 100 \ --test_step 400 \ --ckpt_save_step 50 \ --embedding_size 128 \ --weight_decay 0.0005 \ --eps 1e-6
@4kssoft Do you generate your own dictionnary? I don't really understand the part "Generate your own dictionary with main_build_dict.py / main_data_tokenizer.py". Can you explain how to apply this process on own dataset? Thanks
Also, to what the ckpt_path argument refers to?
from cutie.
Hello @vsymbol
can you please give brief about how to generate the texts and corresponding bounding boxes & manually labelling each text and their bounding box
Which tools we have use for manually labelling
from cutie.
@4kssoft Thanks for the labeling video. Does your software export in the format required by CUTIE (json template you provided) or you have to run explicit post processing ?
In the json example you provided, what does "key_id" and "key_value" represent ? all of them look empty.
from cutie.
Hello @vsymbol
can you please give brief about how to generate the texts and corresponding bounding boxes & manually labelling each text and their bounding box
Which tools we have use for manually labelling
Apply any OCR tool that help you detecting and recognizing words in the scanned document image.
For example, refer to what @4kssoft has done to the document image and generated a .json file with position and text of the image.
https://github.com/4kssoft/CUTIE/blob/master/invoice_data/Faktura1.pdf_0.json
from cutie.
Did someone able to train and test the model? I couldn't find how to predict on new data .
from cutie.
Did someone able to train and test the model? I couldn't find how to predict on new data .
I'm struggling with it. So far I was able to create the .json-files, with the solution of hhien's code.
I anyone succeeded, I'm thankful for any recommendation on how to train the model.
from cutie.
Please tell me if this is the correct data format?
Format:
file_name.json
{ "global_attributes": { "file_id": "$file_name" }, "fields":[ { "field_name": "$class_name", "key_id": [], "key_text": [], "value_id": [$word_id], "value_text": "$word_text" },... ], "text_boxes":[ { "id": $word_id, "bbox": [$word_x_min, $word_y_min, $word_x_max, $word_y_max], "text": "$word_text" },... ] }
Example:
file1.json
{ "global_attributes": { "file_id": "file1.jpg" }, "fields":[ { "field_name": "class1", "key_id": [], "key_text": [], "value_id": [1], "value_text": "sample1" }, { "field_name": "class1", "key_id": [], "key_text": [], "value_id": [2], "value_text": "sample2" }, { "field_name": "class2", "key_id": [], "key_text": [], "value_id": [3], "value_text": "sample3" } ], "text_boxes":[ { "id": 1, "bbox": [10, 10, 50, 20], "text": "sample1" }, { "id": 2, "bbox": [55, 10, 100, 20], "text": "sample2" }, { "id": 3, "bbox": [50, 30, 100, 40], "text": "sample3" } ] }Or maybe the correct format should look like this
Example 2:
file1.json
{ "global_attributes": { "file_id": "file1.jpg" }, "fields":[ { "field_name": "class1", "key_id": [], "key_text": [], "value_id": [1, 2], "value_text": ["sample1", "sample2"] }, { "field_name": "class2", "key_id": [], "key_text": [], "value_id": [3], "value_text": ["sample3"] } ], "text_boxes":[ { "id": 1, "bbox": [10, 10, 50, 20], "text": "sample1" }, { "id": 2, "bbox": [55, 10, 100, 20], "text": "sample2" }, { "id": 3, "bbox": [50, 30, 100, 40], "text": "sample3" } ] }
what are bbox entries? x1,y1,widht,height? or x1,y1(top left),x2,y2(bottom right)
from cutie.
i have created the json files in the required format. i have 400 invoices data. the main_train_json.py gets killed because it utilises all the RAM. has anyone faced this issue? I have 16 gb of ram.
from cutie.
Anyone pls share the inference script?
from cutie.
Related Issues (15)
- Unable to run the main_evaluate_json file HOT 5
- Predict and Output HOT 15
- Dictionary and vocab generation HOT 3
- Training and predicting using model
- Apply model on SROIE Dataset HOT 1
- Which labelling or annoation tool you are using? HOT 4
- Creating input json file for SROIE dataset HOT 3
- Tensorflow 2
- Regarding Saved model
- How to predict each line item in an invoice?
- I am new to Deep Learning can anyone help in getting started with this model
- Input format of data HOT 1
- grid_label minus values ? HOT 12
- Unable to obtain Training Statistic HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cutie.