juditacs / deep-morphology Goto Github PK

Dataset

• __len__: first dimension of the first matrix
• use namedtuples everywhere

Models

• the encoder is the same in every seq2seq model, move it to a separate file and import it everywhere
• the decoder is pretty similar in a few models, base class?
• should attention mechanism be in a separate class?
  • pro: can use different attention variations easily

Some parameters are duplicated.

If the encoder and the decoder has a different hidden size, the hidden state should be projected. Should it be projected even if they're the same size?

Similar to train_many but no randomness.

List of configs

Python or YAML?

Python. It allows creating configurations in a flexible way.

Implementation

Take a base configuration and override certain parameters. Should be implemented as a generator function. A single Python file which defines this generator should be the parameter of the runner script..

Used in many places, move it to a common class.

Subclasses of BaseDataset should only implement printing one sample, not the whole dataset.

Some of the neural modules take a config object, others take parameters separately. This should be more consistent, for example

1. top-level modules should take a single config object and a dataset object. Top-level modules are the ones that can be specified in config.yaml
2. other modules should take each parameter separately. The reason for this is that they are designed to be reusable blocks for other modules.

Hi

Can't run toy example as there are no toy.yaml in root/config and root/data dir.
Are these in an earlier PR / diff repo? Ta

This is the case for defaults. It should be the case for those read from config.yaml

Currently Luong's attention with a projection matrix is the only attention variant supported. Should I add more variants?

this should be handled by the `CUDA_VISIBLE_DEVICES˙ shell variable.

Many errors, fix them.

Run toy training for every model.

examples: commit hash, random seed

Be careful not to break train_replica

Add debug switch. Only print a warning if there are modifications in the working directory.