The backend to PyJet is currently so small, it does not need to be in its own folder. It should be moved into a python file named "backend.py".

Right now all operations on a DatasetGenerator are complicated with ImageAugmentor and GeneratorEnqueuers (and possibly more generators to come) since they do not share the same attributes of a DatasetGenerator. These classes need to be modified to allow their use in any place a normal DatasetGenerator might be used.

This might be achieved by subclassing the DatasetGenerator or creating a super class. This can also be achieved by creating matching attributes for GeneratorEnqueuer and ImageAugmentor defined by the attributes of DatasetGenerator.

Ideally, changing DatasetGenerator should not require changing any solution code for this issue.

In order to be able to seamlessly swap pyjet models with keras models, pyjet needs to change to use the same API for training and interact the same way with training related keras objects (e.g. callbacks). In order to do so, the following needs to happen:

• Write a compile method for SLModel to pass in loss functions, optimizers, and metrics into
• Support multi-input, multi-loss, and multi-output in the same fashion as keras
• Make fit_generator create a training log for callbacks just like Keras
• Modify fit_generator to return a history object
• Replace save_state and load_state with save_weights and load_weights respectively
• Create save and load_model just like with keras
• Change impelentable method for the forward pass of the network to call not forward to use forward for some forward pass overhead
• Create non-generator equivalents (i.e. create Keras's fit, evaluate, and predict methods)

Most of the current PyJet code was hacked together in one weekend without testing. Major, hidden bugs do exist. Having increased testing will not only uncover these bugs, but also increase the stability and organization of the project.

Goal: Have 95% test coverage before adding any new features.

Currently loss functions and auxiliary loss functions are supported, but the setup is very clunky. One main loss function is supported, but the input must be stored in self.loss_in in the network forward() method. Auxiliary losses are appended to the self.aux_loss_in and these losses must be input into the fit_generator method in the exact same order their arguments were appended to self.aux_loss_in. Currently, assigning various optimizers to different losses is not supported.

Some avenues to improve functionality

• Use some kind of dictionary under the hood to link up losses by name with their inputs.
• Have some methods that abstract the use of this dictionary for users to use when creating network. Something like

add_loss_fn(self, name, loss_fn, *inputs)

• Store some kind of similar dictionary to figure out which optimizers go to which losses. This can be input into the fit_generator
• Consider making a compile method that will build this stuff before calling fit_generator. This will allow us to mimic keras better. This method compile will be able to to be called any number of times to change the loss + optimizer dictionaries.

When a program running a generator enqueuer is interrupted or finishes, the termination of the generator enqueuer will hang. This is most likely due to imporperly terminating the threads and some being stuck running or in deadlock during termination.