Move backwards step into model about padertorch HOT 1 OPEN

However, we have to consider the implications for the Hook post_step, which is at the moment called after train_step but before the backwards step.

This is currently done to decrease the memory consumption (i.e. after the post step we can delete the input, and the review).

Another point to consider is, that the multi gpu source code must be changed.
I don't know if calling backward in a thread is allowed and recommended in pytorch.

I would say, we plan to implement it, when we see a demand for it.

A possible workaround (for those that need it now):

• Use single GPU
• Just call backward inside the model
• Return the last loss and the trainer calls backward on this loss

The code wouldn't be pretty, but it should do the task.

from padertorch.