smacawi / tweet-classifier Goto Github PK

• Number of oov words (from GlovE and Word2Vec)
• PRF metrics (All events, similar event, one event) (comparison of models - BERT, Glove RNN, Word2vec RNN)

Include library versions where applicable (e.g. allennlp) and add any missing requirements that prevent replicating the experiments.

Is your feature request related to a problem? Please describe.
Social media text has a lot of non-standard English, leading to a lot of out-of-vocabulary words.
Subword tokenization algorithms such as byte-pair encoding can help with this issue, as well as bypass the need for text normalization which can lose a lot of information.

Describe the solution you'd like
Implement subword tokenization through AllenNLP library and evaluate it's impact on tweet classification.

Describe alternatives you've considered

• Word-level tokenization + normalization: not ideal due to reasons outlined above.
• Character-level tokenization: limited representational capabilities.

Additional context
NA

• Optimizer (Adam LR 0.5e-3, 1e-3, 0.5e-2)
• Final layer (GRU, Bi-LSTM, Sent-pool)
• Batch-size (32, 64, 128)
  AlleNLP BERT implementation:
  https://medium.com/analytics-vidhya/fine-tuning-bert-with-allennlp-7459119b736c

• specify steps for environment setup;
• specify steps for training;
• include link to allennlp documentation/examples for adequate version;
• specify twitter-scraper dependency
• specify steps for inference/deployment;