How to evaluate the Market1501 model? about cosine_metric_learning HOT 4 CLOSED

There is a section on evaluation in the README. Can you be more specific with what problems you encounter?

from cosine_metric_learning.

@nwojke In the README, we just find train and test method ,but we can not evaluate the model.In other words , how to get the accuracy about the Market1501 model?

from cosine_metric_learning.

We used the original dataset SDK for final model evaluation, with the intention to use the exact same software that others used for the reported numbers. Therefore, you need to export features of the test split (can be done with provided scripts in this repository) and then evaluate performance using software of the dataset (cannot be done with provided scripts). I am copying over the part which covers this procedure from the README. Let me know if anything remains unclear.

--

Final model testing has been carried out using evaluation software provided by
the dataset authors. The training scripts can be used to write features of the
test split. The following command exports MARS test features to
./MARS-evaluation-master/feat_test.mat

python train_mars.py \
    --mode=export \
    --dataset_dir=./MARS-evaluation-master \
    --loss_mode=cosine-softmax .\
    --restore_path=PATH_TO_CHECKPOINT

where PATH_TO_CHECKPOINT the checkpoint file to evaluate. Note that the
evaluation script needs minor adjustments to apply the cosine similarity metric.
More precisely, change the feature computation in
utils/process_box_features.m to average pooling (line 8) and apply
a re-normalization at the end of the file. The modified file should look like
this:

function video_feat = process_box_feat(box_feat, video_info)

nVideo = size(video_info, 1);
video_feat = zeros(size(box_feat, 1), nVideo);
for n = 1:nVideo
    feature_set = box_feat(:, video_info(n, 1):video_info(n, 2));
%    video_feat(:, n) = max(feature_set, [], 2); % max pooling 
     video_feat(:, n) = mean(feature_set, 2); % avg pooling
end

%%% normalize train and test features
sum_val = sqrt(sum(video_feat.^2));
for n = 1:size(video_feat, 1)
    video_feat(n, :) = video_feat(n, :)./sum_val;
end

The Market1501 script contains a similar export functionality which can be
applied in the same way as described for MARS:

python train_market1501.py \
    --mode=export \
    --dataset_dir=./Market-1501-v15.09.15/
    --sdk_dir=./Market-1501_baseline-v16.01.14/
    --loss_mode=cosine-softmax \
    --restore_path=PATH_TO_CHECKPOINT

This command creates ./Market-1501_baseline-v16.01.14/feat_query.mat and
./Market-1501_baseline-v16.01.14/feat_test.mat to be used with the
Market1501 evaluation code.

from cosine_metric_learning.

I am closing this issue due to inactivity. Re-open if the problem is not solved.

from cosine_metric_learning.