How to solve the model saturation? about domainadaptivereid HOT 2 CLOSED

Do you mean that the validation accuracy drops when we set the training iterations too large?

I think this is inevitable since the experiments are under an unsupervised setting. All we can do is selecting a best model during training according to the validation results, but such manner is actually cheating since we are supposed to have no idea about any labeling info on the target dataset. But, many papers are doing like this and I don't know why everyone in this field thinks it is okay. The same 'convention' also exists for other unsupervised domain adaptation tasks, such as segmentation. You can simply check some papers (even CVPR oral) that released their code and you will find that they are selecting the best model according to the validation.

The recent studies (papers) of unsupervised domain adaptation are useless to real practical problems, mainly due to their confusing settings (They claimed labelling is costly, but still using the labeled validation data). In fact, we (i.e., the group I am in at Horizon Robotics) do not study such unsupervised settings anymore.

from domainadaptivereid.

I got your idea.
The problem or phenomenon you pointed out is prevalent in the unsupervised learning problem settings. Also, I don't realize this before and it does obviously violate the basic ML rule because of using the validation set during training.
Thanks for this kindly remind and reply.

from domainadaptivereid.