About rvl_cdip_dataset.csv about docformer HOT 6 CLOSED

Hi there,

I have provided an answer to this question, here

And, for the usage of the docformer (i.e to use the DocFormer model), I have also provided a sample notebook, which describes how to train docformer for the purpose of Mask Language Modeling, and with some tweaks, you can extend it to multiple tasks.

Here, is the link for the same link to notebook. Please ignore that, file i.e dataset_creation_for_docformer.py, as it was a sample file, and we would remove it shortly.

Do feel free to ask, for more queries.

Regards,
Akarsh

from docformer.

Thanks for the reply Akarsh. I want to use the docformer code for the classification problem on rvl-cdip dataset.

from docformer.

For that, the things are simple, just have a look at the MLM colab notebook, and then make the argument used for masking the input ids to be false, and in the modelling part, change the dimension of the last layer i.e the classification layer to be equal to the num of classes, and then you can go for training. However, you need to make sure, you have the OCR for RVL-CDIP since, it is a large dataser, else it would take time for generating OCR.

Feel free to ping for more help.

Regards,

from docformer.

Thanks for the reply. I will try this and get back to you.

from docformer.

Hello author:
How to understand the output['ir'] in "criterion2(output['ir'],labels2)" at train_accelerator_mlm_ir.py, it comes from a decoder or other? If the answer is a decoder, can tell us how it was written? In addition, the label2 is the resized_scaled_img(e.g., the pixel is 384*500)? Thans!

from docformer.

Actually, the architecture of the decoder is not mentioned, so earlier I implemented just a simple conv->deconv architecture, but I think, you would have to implement your own decoder, as per your usage. Just take the output of the encoder of DocFormer, and then pass it through Conv, and then Upsample, and I hope that works. Do let me know if there is something more. And yes, earlier the size of the image was 224 x 224 but, now it is 500 x 384

from docformer.