erikatlotic / spanner Goto Github PK

Overview | Demo | Installation | Preprocessing | Prepare Models | Running | System Combination | Bib

This repository contains the code for our paper SpanNER: Named EntityRe-/Recognition as Span Prediction (ACL 2021).

The model designed in this work has been deployed into ExplainaBoard.

We investigate complementary advantages of systems based on different paradigms: span prediction model and sequence labeling framework. We then reveal that span prediction, simultaneously, can serve as a system combiner to re-recognize named entities from different systems’ outputs. We experimentally implement 154 systems on 11 datasets, covering three languages. Comprehensive results show the effectiveness of span prediction models that serve as base NER systems and system combiners.

We deploy SpanNER into the ExplainaBoard.

• python3
• PyTorch
• pytorch-lightning

Run the following script to install the dependencies,

pip3 install -r requirements.txt

The dataset needs to be preprocessed, before running the model. We provide dataprocess/bio2spannerformat.py for reference, which gives the CoNLL-2003 as an example. First, you need to download datasets, and then convert them into BIO2 tagging format. We provided the CoNLL-2003 dataset with BIO format in the data/conll03_bio folder and its preprocessed format dataset in the data/conll03 folder.

The download links of the datasets used in this work are shown as follows:

• CoNLL-2003
• CoNLL-2002
• OntoNotes 5.0
• WNUT-2016
• WNUT-2017

For English Datasets, we use BERT-Large.

For Dutch and Spanish Datasets, we use BERT-Multilingual-Base.

Here, we give CoNLL-2003 as an example. You may need to change the DATA_DIR, PRETRAINED, dataname, and n_class to your own dataset path, pre-trained model path, dataset name, and the number of labels in the dataset, respectively.

./run_conll03_spanner.sh

We provided 12 base models (result-files) of the CoNLL-2003 dataset in combination/results. More base models (result-files) can be downloaded from ExplainaBoard-download.

Put your different base models (result-files) in the data/results folder, then run:

python comb_voting.py

Here, we provided four system combination methods, including:

• SpanNER,
• Majority voting (VM),
• Weighted voting based on overall F1-score (VOF1),
• Weighted voting based on class F1-score (VCF1).

@article{fu2021spanner,
  title={SpanNer: Named Entity Re-/Recognition as Span Prediction},
  author={Fu, Jinlan and Huang, Xuanjing and Liu, Pengfei},
  journal={arXiv preprint arXiv:2106.00641},
  year={2021}
}