yueatsprograms / stochastic_depth Goto Github PK

I'm trying to train the 1202 layer network. I can't fit the network into one Titanx with batch size 128 (even with optnet).
I tried to use DataParallelTable as in fb.resnet.torch, however, due to some unknown reason, I can't make it work.
Can you provide your multi-GPU version of the code?

Hi,

Just to let you know that you could use https://github.com/fmassa/optimize-net for automatically optimizing for memory the 1202-layer resnet architecture.

Thanks for releasing the code. I am using one TitanX to reproduce the results of Stochastic Depth for cifar10, but the val error does not decrease since epoch 1, as shown below:
resultFolder : "results-cifar10/"
device : 2
dataset : "cifar10"
N : 18
batchSize : 128
augmentation : true
maxEpochs : 500
deathRate : 0.5
deathMode : "lin_decay"
}
Training set size: 45000
Validation set size: 5000
Test set size: 10000
Building model...
Training...
Epoch Valid. err Test err Training time
Epoch 1: 89.74% 90.00% 51s
Epoch 2: 90.02% 90.00% 51s
Epoch 3: 89.74% 90.00% 51s
Epoch 4: 89.74% 90.00% 51s
Epoch 5: 89.74% 90.00% 51s
Epoch 6: 89.74% 90.00% 51s
Epoch 7: 89.74% 90.00% 51s
Epoch 8: 89.74% 90.00% 51s
Epoch 9: 89.74% 90.00% 51s
Epoch 10: 89.74% 90.00% 51s
Epoch 11: 89.74% 90.00% 51s
Epoch 12: 89.74% 90.00% 51s
Epoch 13: 89.74% 90.00% 51s
Epoch 14: 89.74% 90.00% 51s
I am wonderring if any configuration is wrong.