benjamintdk / dsai_proj Goto Github PK

Possibly to visualize a single example of the dataset.

Fix formatting of descriptions.

Need to process year, month and day further.

Year --> possibly binning
Month --> divide by 12
Day --> divide by total no. of days of that month

Download actual dataset for visualization to fix CI issues.

Other things to consider:

• Plot word counts across binned years? Show relevance of years
• Plots needs to be more clearly defined and elaborated on in terms of their use; axis labels can improve too. Flow of thought can be better expressed.

1. Can we use transformer architecture with pretrained encoders and decoders already? Reduces headache of deciding how many decoders to use. --> Stick with untrained decoders, use [SEP] token as starting token.

2. How to invert the word embedding layer to use as final output layer instead of simple linear layer? If cannot, do we use vanilla embedding or linear layer? --> Just stick with linear layer for simplicity, followed by softmax.

3. How to generate final output sequence? Do we need a for loop? --> Yes we do based on source code. Set to max length. This should be set when creating the datasets too.