- We utilized CRF classifiers from Stanford CoreNLP for the Event and Temporal Span identification tasks of TempEval-3. The aim of the TempEval series was to advance research on temporal information processing. This project was conducted as part of graduate level course-work in Machine Learning (CS 613) taught at Drexel University in Fall, 2016.
- We performed feature engineering as suggested in the Stanford NER system and utilized word-level, char-level and n-gram level features alongside certain positional features.
- We performed an ablation with respect to the size of the training data upto 2.5k train documents.
- Precision was stagnant after a mere 50 training samples.
- Controlling for these false positives, we found Recall to increase on a logscale with additional documents in steps of constant size (50 in our case).
- We performed a qualitative assessment of the TempEval-3 task (News domain) and compared it to the SemEval-2016 task which was based on documents from the Clinical domain.
- Temporal spans were easier to identify in TempEval-3 since News contains more absolute expressions such as
Last May,2010,eight yearsetc. On the other hand, the Clinical domain is much harder for Temporal span identification due to complex relative expressions such asa day before surgeryetc. - Conversely, the Clinical domain is easier for Event extraction due to the higher density of standard events and operating procedures found in such a corpus.
- Temporal spans were easier to identify in TempEval-3 since News contains more absolute expressions such as
- Kindly refer to our paper for further detail.
- src contains source code and instructions to install libraries (CoreNLP), datasets and existing models
- paper is our conference-style paper generated using Latex
- presentation is our final presentation which summarizes our key experiments
- Our code requires Ubuntu Linux (or any comparable POSIX compliant environment) to run
- The codebase uses
Java 8andPython 2.7, so both of these languages must be installed - Our code can be run with default parameters from
srcusingpython control.py
- It also supports some command line flags:-
-pre_train_skipskips preprocessing of the TimeML training set into COL format. Use if COL files are already present-train_skipskips training and creation of the NER model. Since this process can require hours, it is advisable to use a pretrained model for inference-test_skipskips the testing process. This can be used if only model training is needed-train_n <number>allows to train on a sample of randomly chosen training files, since training on the entire dataset is time consuming
- It is advisable to use a machine with at least 8 GB of RAM. It will run with less memory, but performance will suffer.
- Our project (+CoreNLP) is hardcoded to use 4 GB of RAM, but this can be changed. Inability to allocate at least the specified memory 8 GB will cause an OS crash.
- Our project requires approximately 500 MB of disk space, but allowing at least 1 GB is advisable.
- Stanford NER licensed under the GNU GPL (v2 or later)
- Stanford CoreNLP licensed under the GNU GPL (v3 or later)
https://www.dropbox.com/s/6uylvx80ece0zfr/Israney%2C%20Ramakrishna%20-%20Temporal%20Expression%20and%20Event%20Extraction.zip?dl=0
- Our last trained model used train data size = 100.
- To train model with full training set, run:
python control.py
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent