This Code describes how to seed up the training of CNN (Convolutional Neural Network), and also shows whether the results change. there are some my Chinese communication websites such as CSDN and Quora (Chinese)-Zhihu where I explain this code.

CSDN

Quora (Chinese)-Zhihu

GPU is RTX 2060 (6G), So the experiment didn't work well. (No Money)

This is just a Demo of how to speed up training on a single and multi-GPU (simulation).

AlexNet was taken as the network architecture(image size required for input was 227x227x3) , CIFAR10 as the datset, Adamw as the gradient descent function, and learning rate mechanism as ReduceLROnPlateau for example.

You should create your own data(dataset), checkpoint(model save) and tenshorboard(loss visualization) file package.

1.The results of the model without DP or DDP

Run Training Code: python Normal/train_XXX.py or train.py

Run Evaluation Code: python Normal/eval.py

2.The results of the model with DP

Run Training Code: python DP/train_XXX.py or train.py

Run Evaluation Code: python DP/eval.py

3.The results of the model with DDP

Run Training Code: python DDP/train_XXX.py or train.py

Run Evaluation Code: python DDP/eval.py