transfer learning - load pretrained model about densenet HOT 5 CLOSED

@idimitriadis That's my fault.

In the earlier model, I did not follow the exact methodology of the paper (did not use weight regularization, did not use He et al initialization). In addition to these mistakes, I also added the initial convolution with bias, which was not according to the paper.

That is why the error is occuring. The model assumes only convolution weights will be provided (no bias), but the weights file has the weights for the bias as well. Simply removing the bias term in the weights file is wrong, since the prediction accuracy drops dramatically.

That is why I am retraining the DenseNet 40-12 model on CIFAR 10 again and this will have the correct implementation. Please wait a few days for the weights.

from densenet.

thank you for your immediate response!
Keep up the good work!

from densenet.

@idimitriadis Just uploaded the final model. It gets 94.45% accuracy compared to 94.76 % according to the paper for the DenseNet 40-12 model.

I used Adam with a low learning rate of 1e-4 and trained for 150 epochs only, since each iteration takes over 10 minutes for me, so that may be the reason that it is scoring slightly less.

I've also begun implementing the faster version of DenseNet that the authors are using now. It significantly speeds up training (6 minutes per epoch), but I still have to train on the CIFAR-10 to see if the implementation is correct.

Edit:
Trained the DenseNet fast model as well, and while it trains faster, it reaches its best performance after many more epochs. Perhaps this is due to an implementation issue (there is no ScaleLayer and BiasLayer specifically in Keras, unlike Lasagne). After approximately 250 epochs, it scored 94.29 % accuracy.

from densenet.

Hi, have you tried to train with imagenet dataset?

from densenet.

@MrXu No I haven't tried training on ImageNet. I don't have the GPU processing power to train on ImageNet.

However, there are now parameters to create DenseNet BC models which were used to train DenseNet on ImageNet

from densenet.