Hey all,
I'd like to experiment with those pre-trained models,
Where can I get them?

Thank you 👍

Thanks for your great work,
may I ask where can I find the WCC dataset for the table type classification task?
Thanks!

It seems that the websheet dataset used in the Cell Type Classification cannot be accessed. Could you also share the dataset?

I've downloaded the dataset but it needs some pre-processing to get it to your format, as in the sample you provide in the repo.
Do you have the scripts for this process?

Hi,
I've been trying to do ctc_finetune but on other json files than deex.json presented in your Readme and I am wondering how you can get the labels to do this task? In deex.json, label_matrix is already a part of a file, and therefore, I cannot reproduce it to other json files, including those you have in your repository like wiki-tables-samples.json.
I would be very grateful for an advice :)

Can you please upload the pip requirements of the project? Thanks

I'd like to make a simple inference on few excel sheets of mine,
I have noticed the json files holds some keys, which are not mentioned in the data originated from wiki.

T: cell text 
V: cell value
NS: number string
DT: data type (internally stored in spreadsheets, text=0,number=1,data_time=2,percentage=3,currency=4,others=5)
HF: if has formula
A1: formula string with A1 form (absolute cell reference)
R1: formula string with R1C1 form (relative cell reference)
LB: if has left border
TB: if has top border
BB: if has bottom border
RB: if has right border
BC: if has non-white background color
FC: if has non-black font color
FB: if has font bold
I: if has font italic
HA: horizontal alignment (center=0, center_across_selection=1,distributed=2,fill=3,general=4,justify=5,left=6,right=7)
VA: vertical alignment (top=0,center=1,bottom=2,justify=3,distributed=4)

@HaoAreYuDong, I couldn't find the script which generates those annotations/keys for the corresponding excel files?
Can you please share it.

Thank you very much,
I appreciate the assistance!

Hi, I am trying to pretrain a model based on your spreadsheet sample. However, the code rejects the sample for some reason.
The same happens for some manual spreadsheet samples that I've created.

I believe this happens due to lines 681-682 in tokenizer.py:

if (max(top_pos_list[icell]) == -1) or (max(left_pos_list[icell]) == -1):
    return None

But I can not understand thoroughly what's happening behind these conditions for these lists.
Can you please explain why your given spreadsheet sample is not working?

Hi,@HaoAreYuDong,The link to A preprocessed dataset of DeEx is not working, can you put a brand new link in?

The link I refer to is as follows:
https://drive.google.com/file/d/1xJkq2DQciWvndhgm0aHZXMqzIWSan9z9/view?usp=sharing

Hi there,
do you plan to release code (and data, or example data) for the table type classification downstream task?
Thanks!

Hi and thanks for uploading your code repo.

How can someone preprocess their spreadsheet and generate a JSON for it according to your format?
https://github.com/microsoft/TUTA_table_understanding/blob/main/data/pretrain/spreadsheet/spreadsheet-sample.json

I want to convert these files enron_{train/dev/test}.pt to enron_{train/test}_fortap_input.pt. Is there a script to do so?

I've downloaded the dataset but it needs some pre-processing to get it to your JSON format, as in the sample you provide in the repo.
Do you have the scripts for this process?

@HaoAreYuDong

Could you give me the three datasets WebSheet, SAUS, and CIUS for "Cell Type Classification"?

I would like to have the data after conversion to TUTA input format.

Could you please tell me the command line arguments when you fine-tune with the TUTA-implicit model?

"--hidden_size", type=int, default=768
"--intermediate_size", type=int, default=3072
"--magnitude_size", type=int, default=10
"--precision_size", type=int, default=10
"--top_digit_size", type=int, default=10
"--low_digit_size", type=int, default=10
"--max_cell_length", type=int, default=16
"--row_size", type=int, default=2560
"--column_size", type=int, default=2560
"--tree_depth", type=int, default=4
"--node_degree", type=str, default="32,32,64,256"
"--attention_distance", type=int, default=2
"--attention_step", type=int, default=0
"--num_attention_heads", type=int, default=12
"--num_encoder_layers", type=int, default=12
"--hidden_dropout_prob", type=int, default=0.1
"--attention_dropout_prob", type=int, default=0.1
"--layer_norm_eps", type=float, default=1e-6
"--hidden_act", type=str, default="gelu"
"--learning_rate", type=float, default=8e-6

"--max_seq_len", type=int, default=512
"--max_cell_num", type=int, default=256
"--text_threshold", type=float, default=0.5
"--value_threshold", type=float, default=0.1
"--clc_rate", type=float, default=0.3
"--wcm_rate", type=float, default=0.3
"--add_separate", type=bool, default=True
"--num_ctc_type", type=int, default=6

"--attn_method", type=str, default="add", choices=["max", "add"]
"--hier_or_flat", type=str, default="both", choices=["hier", "flat", "both"]
"--org_or_weigh", type=str, default="original", choices=["original", "weighted"]
"--num_format_feature", type=int, default=11
"--sep_or_tok", type=int, default=0, choices=[0, 1]
"--sep_weight", type=float, default=0.0
"--aggregator", type=str, default="sum", choices=["sum", "avg"]

"--target", type=str, default="tuta"

"--batch_size", type=int, default=2
"--report_steps", type=int, default=200
"--epochs_num", type=int, default=40
"--dataset_num", type=int, default=1
"--early_stopping_bound", type=int, default=100