cascadetsrdet's Introduction

Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images

Paper Link

Requirements

This codebase is built on top of Detectron2. Follow the instructions here to install Detectron2.

Datasets and Pretrained Model

Dataset	Weights
PubTables1M	PubTables1M
FinTabNet	FinTabNet
SciTSR	SciTSR

Configuration and Training

Inference and Evaluation

Check the inference.py and test.sh for the inference. A sample inference command could be:

python inference.py --mode recognize --structure_config_path <path of config.yaml> --structure_model_path <path of weight> --structure_device cuda --image_dir <dir of table images> --out_dir <output dir> --html --visualize --csv --crop_padding 0

Check the teds.py for calculating the TEDS score.

Citing

Please cite our work if you think it is helpful:

@article{xiao2023rethinking,
  title={Rethinking Detection Based Table Structure Recognition for Visually Rich Documents},
  author={Xiao, Bin and Simsek, Murat and Kantarci, Burak and Alkheir, Ala Abu},
  journal={arXiv preprint arXiv:2312.00699},
  year={2023}
}

Acknowledgement

This project heavily relys on Table-Transformer, especially for the post-processing part. We thank the authors for sharing their implementations and related resources.

cascadetsrdet's People

Contributors

Stargazers

Watchers

cascadetsrdet's Issues

Model loading warnings

Hello!

During loading of PubTables1M model I see the following messages:

Creating inference pipeline
Some model parameters or buffers are not found in the checkpoint:
backbone.bottom_up.res2.0.conv1.norm.num_batches_tracked
backbone.bottom_up.res2.0.conv2.norm.num_batches_tracked
backbone.bottom_up.res2.0.conv3.norm.num_batches_tracked
backbone.bottom_up.res2.0.shortcut.norm.num_batches_tracked
backbone.bottom_up.res2.1.conv1.norm.num_batches_tracked
backbone.bottom_up.res2.1.conv2.norm.num_batches_tracked
backbone.bottom_up.res2.1.conv3.norm.num_batches_tracked
backbone.bottom_up.res2.2.conv1.norm.num_batches_tracked
backbone.bottom_up.res2.2.conv2.norm.num_batches_tracked
backbone.bottom_up.res2.2.conv3.norm.num_batches_tracked
backbone.bottom_up.stem.conv1.norm.num_batches_tracked
The checkpoint state_dict contains keys that are not used by the model:
  backbone.bottom_up.res3.4.spatial_gating_unit.conv0.{bias, weight}
  backbone.bottom_up.res4.6.spatial_gating_unit.conv0.{bias, weight}
  backbone.bottom_up.res5.3.spatial_gating_unit.conv0.{bias, weight}

Is this behavior correct?

Recommend Projects