SemEval 2019 - Task 6 - Identifying and Categorizing Offensive Language in Social Media

A system to classifiy a tweet as either offensive or not offensive (Sub-task A) and further classifies offensive tweets into categories (Sub-tasks B – C). Some sort of grid search approach is taken where multiple techniques for preprocessing, feature extraction and classification are implemented and combinations of them all are tried to achieve the best model for the given dataset.

• Sub-task A - Offensive language identification; [Offensive: OFF, Not Offensive: NOT]
• Sub-task B - Automatic categorization of offense types; [Targted: TIN, Untargeted: UNT]
• Sub-task C: Offense target identification. [Individual: IND, Group: GRP, Other: OTH]

Tokenization, Stopwords Removal, Lemmatizaion, Stemming

TFIDF, Count, Word2Vec, GloVe, fastText

KNN, Naïve Bayes, SVM, Decision Trees, Random Forest, Logistic Regression, MLP, Adaboost, Bagging

LSTM, 1-D CNN

• Install requiremetns using pip3 install -r requirements.txt

• python3 tune.py to do a complete search on all combinations of prepocessing, vectorization and non-deep classification techniques while tuning the classifiers hyper-params.

• python3 train.py to train one of the deeplearning models.