Comments (6)
Hi there,
I have provided an answer to this question, here
And, for the usage of the docformer (i.e to use the DocFormer model), I have also provided a sample notebook, which describes how to train docformer for the purpose of Mask Language Modeling, and with some tweaks, you can extend it to multiple tasks.
Here, is the link for the same link to notebook. Please ignore that, file i.e dataset_creation_for_docformer.py
, as it was a sample file, and we would remove it shortly.
Do feel free to ask, for more queries.
Regards,
Akarsh
from docformer.
Thanks for the reply Akarsh. I want to use the docformer code for the classification problem on rvl-cdip dataset.
from docformer.
For that, the things are simple, just have a look at the MLM colab notebook, and then make the argument used for masking the input ids to be false, and in the modelling part, change the dimension of the last layer i.e the classification layer to be equal to the num of classes, and then you can go for training. However, you need to make sure, you have the OCR for RVL-CDIP since, it is a large dataser, else it would take time for generating OCR.
Feel free to ping for more help.
Regards,
from docformer.
Thanks for the reply. I will try this and get back to you.
from docformer.
Hello author:
How to understand the output['ir'] in "criterion2(output['ir'],labels2)" at train_accelerator_mlm_ir.py, it comes from a decoder or other? If the answer is a decoder, can tell us how it was written? In addition, the label2 is the resized_scaled_img(e.g., the pixel is 384*500)? Thans!
from docformer.
Actually, the architecture of the decoder is not mentioned, so earlier I implemented just a simple conv->deconv architecture, but I think, you would have to implement your own decoder, as per your usage. Just take the output of the encoder of DocFormer, and then pass it through Conv, and then Upsample, and I hope that works. Do let me know if there is something more. And yes, earlier the size of the image was 224 x 224 but, now it is 500 x 384
from docformer.
Related Issues (20)
- Reference code of pre-training tasks HOT 4
- [Errno 2] No such file or directory: 'rvl_cdip_dataset.csv'
- NER task HOT 4
- Predictions are wrong. HOT 1
- can you provide visual question answering task code HOT 1
- Unable to convert model to onnx HOT 3
- DocFormer for Token Classification. HOT 3
- Shape mismatch during sanity check HOT 1
- finetune the Docformer HOT 8
- Pre-trained models HOT 5
- Error in Example: Please provide the bounding box and words or pass the argument "use_ocr" = True HOT 1
- DocFormer for key-value pairs extraction HOT 1
- Using pre-trained models HOT 17
- Inference for token classification. HOT 1
- We can not find DocFormer_For_IR in modeling.py and modeling_l.py HOT 1
- Permission Denied Pretraining Weights HOT 2
- NotImplementedError: Support for `validation_epoch_end` has been removed in v2.0.0. `DocFormer` implements this method HOT 2
- pre-training code tutorial HOT 3
- The pre-trained weight link is invalid HOT 1
- DocFormerv2 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docformer.