A simple implementation a Deep Learning models' training loop built on top of pytorch
with maximal compatibility with that framework in mind.
The pytorch
framework provides a very clean and straightforward interface to build (Deep) Machine Learning models and read the datasets from a persistent storage. So let's use the best features of this great tool and write a set of thin and transparent wrappers on top of it to build a general-purpose training/validation loop that will be able to accept Dataset
and Module
instances, and run training process using modern Deep Learning training techniques.
The project is at the very beginning of its development and lacks many desired features and tests. Therefore, there is a long list of improvements to be implemeted (from must-have to more optional):
- Basic implementation of training loop for CNN-based image classification models
- Simplify training loop instantiation
- More callbacks (early stopping, model saver,
visdom
integration) - Add notebooks with benchmarks and examples
- Make possible to pass plain PyTorch classes and objects directly into the training loop function
- Smoke tests and sanity checks to verify the correctness of training process
- CNN regression
- Adding more examples and applications (Jupyter notebooks)
- Continuous integration is added to the repository
- Benchmarking on "classical" image datasets
- Basic set of image augmentations
- Basic RNN support
- Basic GAN support
- psutil
- numpy
- pandas
- torch
- torchvision
- (dev only) pytest
Please check the following projects (especially, the last one) if you would like to have something that is more suitable for production usage with less manual work and debugging:
-
Ignite โ an official high-level interface for PyTorch
-
Torchsampleโโโa Keras-like wrapper with callbacks, augmentation, and handy utils
-
Skorchโโโa scikit-learn compatible neural network library
-
fastaiโโโa powerful end-to-end solution to train Deep Learning models of various complexity with high accuracy and computation speed
The repository started as an author's attempt to write some simple solution to train an image classifier with modern Deep Learning training techniques as described in this post. It is mostly about learning and implementing interesting algorithms, alongside with robustness and clean code.