Implement the TextRank algorithm according to this paper.
https://web.eecs.umich.edu/~mihalcea/papers/mihalcea.emnlp04.pdf

Swap out SBERT/teacher/student with the GRU model

Set up a Metrics class with functions and a CSV output. I'll add a template to the master branch.

Implement the ELCoM baseline model

Swap out SBERT/teacher/student with the RNN model

Report any bugs and implementation issues here.

Implement Euclidean Similarity in the SBERT model and publish results in #1

Implement Manhattan Similarity in the SBERT model and publish results in #1

Reference Paper:
KL Divergence (Gaussian Mixture Model) Multi-Sense emoji embedding, inspired by this [paper] (https://arxiv.org/abs/1911.06118)

Implement core functionality to start running tests and generating results:

• #18 ELCo Dataloader
• #17 Metrics for Testing
• #21 ELCoM Baseline Model (Unsupervised)
• #22 Teacher-Student Model (Supervised)

Swap out the teacher/student model with the DeText model and report the results in #1

Report any bugs and implementation issues here.

Create a PyTorch Dataset and DataLoader for the ELCo dataset.

Implement Manhattan Similarity in the SBERT model and publish results in #1

Implement the Teacher-Student Model according to the ELCo Defense

Write down progress and statistics here as comments so that everyone can keep track of what's happening.
In the end, we include relevant statistics in our final submission.

Implement the SBERT unsupervised model and perform the following:

• #3 Euclidean Similarity
• #4 Jaccard Similarity
• #5 Manhattan Similarity
• #6 KL Divergence

Create graph embeddings of emojis using graph neural networks.
Paper: https://arxiv.org/abs/2201.06367

Swap out SBERT/ELCoM teacher/ELCoM student with the following models:

• #8 GRU
• #9 LSTM
• #10 RNN
• #11 Transformer
• #12 DeText

Read up on the following papers as necessary:

• #2 SBERT https://arxiv.org/abs/1908.10084
  • #3 #4 #5 Distance Metrics: https://towardsdatascience.com/9-distance-measures-in-data-science-918109d069fa
  • #6 KL Divergence in SBERT: https://arxiv.org/abs/1911.06118
• #7 Model Swaps
  • #8 GRU:
    • https://pytorch.org/docs/stable/generated/torch.nn.GRU.html
    • https://paperswithcode.com/method/gru
  • #9 LSTM:
    • https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html
    • https://paperswithcode.com/method/lstm
  • #10 RNN:
    • https://pytorch.org/docs/stable/generated/torch.nn.RNN.html
    • https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks
  • #11 Transformer:
    • https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html
    • https://jalammar.github.io/illustrated-transformer/
  • #12 DeText https://arxiv.org/pdf/2008.02460.pdf
• #13 Extensions
  • #14 TextRank: https://web.eecs.umich.edu/~mihalcea/papers/mihalcea.emnlp04.pdf
  • #15 Unsupervised Graph Neural Network: https://arxiv.org/abs/2201.06367

Feel free to ask any questions on the methods here.

These are tasks that we most likely won't have time for, but will attempt if we can:

• #6 KL Divergence SBERT
• #11 Transformer Model Swap
• #12 DeText Model Swap
• #14 TextRank Model
• #15 Unsupervised Graph Neural Network

Swap out the teacher/student model with the Transformer architecture and report the results in #1

Report any bugs and implementation issues here.

Implement Jaccard Similarity in the SBERT model and publish results in #1

Swap out SBERT/teacher/student with the LSTM model