The training loss is decreasing while the validation loss is increasing about mae HOT 4 CLOSED

These are the results of logs.

from mae.

Yes, the model is overfitting since cifar10 is a small dataset and we did not perform any data augmentation.

If you wonder why the accuracy is still increasing too, that is because the model is trading accuracy with the cost of perplexity. That is it gets less certain about the right classification, e.g. decrease the probability from 0.9 to 0.6, but get more things right.

from mae.

I see. And where should I extend the dataset by using data augmentations? In the pretrain program or fine-tuning program?
p.s. I'm sorry to ask such questions as this was the first CV program I run.

from mae.

You can add data augmentations when defining the dataset, i.e. changing the transform from here. Something like RandomResizedCrop should be very helpful. But unfortunately, I do not have the computation resource to verify.

from mae.