Twitter named entity extraction for WNUT 2016 http://noisy-text.github.io/2016/ner-shared-task.html and the corresponding workshop paper at WNUT COLING 2016, titled Semi-supervised Named Entity Recognition in noisy-text by Shubhanshu Mishra and Jana Diesner

The dataset used in this repository can bs downloaded from https://github.com/aritter/twitter_nlp/tree/master/data/annotated/wnut16

See Word2Vec.ipynb for details on the original submitted solution for the task.

See Run Experiments.ipynb for the details on the improved system. See Run Experiment.ipynb for the details on the improved system with test data.

The final system is packaged as an API specified in the folder NoisyNLP. More updates will be made to the API in upcoming days. See Run Experiment.ipynb for API usage.

See Updated Gazetteers.ipynb, Extra Gazetteers.ipynb, Download Wikidata.ipynb

See Gen new clusters.ipynb

See Data preprocessing.ipynb

See KerasCharRNN.ipynb, and KerasWordRNN.ipynb

Please cite as:

@INPROCEEDINGS {mishra2016_wnut_ner,
    author    = "Shubhanshu Mishra and Jana Diesner",
    title     = "Semi-supervised Named Entity Recognition in noisy-text",
    booktitle = "Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT)",
    publisher = "The COLING 2016 Organizing Committee",
    pages     = "203-212",
    url       = "http://aclweb.org/anthology/W16-3927",
    year      = "2016",
    month     = "dec"
}