Low accuracy in pretrained model on Market-1501 about mmt HOT 2 CLOSED

Hi,

I have just re-trained my code again to find if there exist any issues in the pre-training stage (./scripts/pretrain.sh dukemtmc market1501 resnet50 1). And the results are satisfactory, i.e. on dukemtmc

Mean AP: 69.5%
CMC Scores:
  top-1          85.0%
  top-5          92.0%
  top-10         94.3%

and on market1501

Mean AP: 29.6%
CMC Scores:
  top-1          58.2%
  top-5          73.5%
  top-10         79.3%

Although 29.6% is not as high as 31.8% (as reported), I don't think it would affect the performance of the following MMT stage. And the reasons for such a slight drop may be due to the fact that (1) randomness of training and (2) the decreased iter number (200->100). The reason for the decreased iter number refers to the below.

As for the phenomenons you provided, I think the value of triplet loss is abnormal. The triplet loss is about 0.024 in my experiments, e.g.

Epoch: [79][100/100]    Time 0.253 (0.277)      Data 0.000 (0.014)      Loss_ce 1.086 (1.076)   Loss_tr 0.037 (0.024)   Prec 100.00% (99.98%)

Maybe you need to check the convergence of the network. Did you modify anywhere of the code?

As for your questions,

1. margin: in the section 3.1 of the paper, the margin=0.5 is for the conventional triplet loss. And here in my pre-training code, we use hard-version softmax-triplet loss (Eq. (6) in the paper), so margin=0 is better.
2. iters: I use iters=200 in the original paper's experiments. And later, I found that iters=100 could achieve similar pre-training performance but with much faster speed than iters=200. So I modify the scripts.

from mmt.

Hi, thank you for your reply.

I have finally found the problem and achieve 30.1%/58.3% on target dataset (Market1501).
As you suggest, the convergence of the network went wrong due to the reset of the initialization.

For some reasons, I had to load the pretrained backbone of ResNet50 from the local (self-defined path). Therefore, I set the parameter "pretrained" as False in Class "models", and manually loaded the pretrained backbone of ResNet50.

I revised "mmt/models/resnet.py" like this for loading the pretrained backbone of ResNet50 from the local
if depth not in ResNet.__factory:
raise KeyError("Unsupported depth:", depth)
resnet = ResNet.__factorydepth
resnet.load_state_dict(torch.load(LOCAL_PATH+ 'resnet50-19c8e357.pth'))

However, when the "pretrained" is set False, it will lead to the reset of the initialization
if not pretrained:
self.reset_params()

Thus, the network will be randomly initialized and not loaded from the pretrained backbone, and finally lead to the poor convergence.

Thanks again for the reply of my questions, it is really helpful.

from mmt.