nvlabs / dg-net Goto Github PK

Hello, In your paper, I think if use appearance space of id I and structure space of id J to generate an image. The ID of generated image should be J. So I think the structure space encodes the ID information.
This loss use the ID of appearance space to determine the ID of generated image.

But this loss use the ID of structure space to determine the ID of generated image.

And you also mentioned in the paper that

When training on images organized in this manner, the discriminative module is forced to learn the fine-grained id-relate attributes (such as hair, hat, bag, body size, and so on) that are independent to clothing.

And obviously the structure space encodes the hair, hat, bag and body size. Therefore, the sutructure space encodes the ID.

This is confusing to use appearance space to discriminative ID. Could you please explain about this.

dear author：
i want to train a model using my dataset,but i do not know how to train a teacher model,can you give me some tips?

Thank you for your code. I try to use MSMT and cuhk03 to train this model, but when I try to enter 'python train.py --config configs/latest.yaml' , command line output an error: ‘size mismatch for classifier.classifier.0.bias: copying a param with shape torch.Size([751]) from checkpoint, the shape in current model is torch.Size([2559]).' I have found out that I must change the param in latest-fp16.yaml and config.yaml(I think config.yaml may be useless in this issue). And I have found out that this error is from /models/best/net_last.pth. The input kinds isn't suitable for this net. Do you have any solution to this or should I change the file?
@layumi

hey there,
here the "outs0" is a feature matrix in shape of 8x1x64x32, and you used it to minus 1 and take the mean of them as loss. what is the principle behind this?

1. When I use another dataset，The generated data is not as much as the original data.For example, my training set is 3G, and I only got 2G generated pictures in the end. What is the reason? Where else to modify?
2. In addition, if the number of training iterations is increased (the default is 100,000), will the quality of the generated images be higher?

I will be grateful if you can answer

I notice that your network code support nsgan and wgan additionally, but I can't run nsgan (maybe because of apex, I'm not sure) straightly and it seems that wgan part is not fully functional. So have you ever tried to use nsgan or wgan to train? And how did it work? Thank you!

Hi @layumi ,
I was a bit confuse when read your code. I can't find appearance encoder and discriminator like your describe in your paper in your code. are they style encoder and MsImageDis in networks.py right ?

Hello,
Thank you for this so modular, easy to understand code.
I have a doubt, as far as i can understand, you are trying to sample images using train_loader_a and train_loader_b, such that images_a and pos_a are of same id, similarly images_b and pos_b are of same id, but images_a and images_b are different id.. Am i correct?
If this is the case, where has the constraint been put that these 2 loaders at run time must load images of different identity? Also there is a method _get_neg_sample in class ReIDFolder. I thought that would be getting used somehow, but I cannot see anywhere that method getting used.
Please help me if i am going wrong anywhere. Your reply would be highly appreciated.

Thanks,
Chaitra

Have you tried cross-dataset testing - training on one dataset, say market1501 and testing on another, say cuhk03 or duke? Have you come across any model with code which has tried this? Will be really grateful to you, thanks in advance.

Hello! Thanks for your excellent job!
I'm now trying to use DG-Net to create dataset. However I noticed that image-saving codes in 'test_folder.py' like this:

im.save(dst_path + '/%03d_%03d_gan%s.jpg'%(label2, label, name[:-4]))

im.save(dst_path2 + '/%03d_%03d_gan%s.jpg'%(label2, label, name[:-4]))
and
image_datasets = datasets.ImageFolder(opts.input_folder, data_transforms)

dataloader_content = torch.utils.data.DataLoader(image_datasets, batch_size=opts.batchsize, shuffle=False, pin_memory=True, num_workers=1)

dataloader_structure = torch.utils.data.DataLoader(image_datasets, batch_size=opts.batchsize, shuffle=True, pin_memory=True, num_workers=1)

image_paths = image_datasets.imgs

So the name of pictures is , for example '001_010_gan0043_c3_xxxxxx.jpg'
Here 0043_c3 is the dataset label of label 010, which represents the structure provider.
I wonder if any provided code exists in the DG-Net code, to help me get the picture name of label 001.(appearance provider)
Or I have to modify my DataLoader to get names XD.
I'd appreciate it if you can give me some help.

I found that when using multi-GPU to tarin, the function write_loss can't work causing it's variable members is empty. And when i use one GPU, it works.

Did i set it up wrong?

Hi,

I was wondering if I can implement this DG-Net in the Nvidia Jetson Nano and I have the followinmg questions:

It is posswible to apply TensorRT to the NN in order to first create an UFF model an the apply TensorRT?
It is possible to add this NN to a deep stream pipeline?

Best regards.

Hello, thank you very much for providing the implementation code of the DG-Net model.
I encountered some problems during the implementation of the project. I will be honored if you can give me some suggestions.

I found the f=f.detch() in the ft_netAB. This causes the vector f to have no gradient. Then the loss_gen_recon_f_* has no no contribution to model parameter update. Is there something wrong with me or something wrong with the code?

I will be grateful if you can give me some suggestions. Thank you！

I checked that torch.nn.DataParallel has been used. I wonder why multi-gpu model won't work. Thanks in advance.

What should I do if I want to switch to another dataset, such as Duke and MSMT17?Does the teacher model need to be retrained?

Hi, thanks for your great work.
I noticed that x_ba is feeded into self.id_a two times. One for teacher loss (only influence id_a) and another for code reconstruction loss and recon id loss (not influence id_a).

Have you tried only forward once and will that hurt the model? I confused about why it's designed in this way.

Why you have sampled the images like this?

x_a, x_a_recon, x_aba, x_ab1, x_b, x_b_recon, x_bab, x_ba1

when the fake image wrt x_a is x_ba1, why you are showing x_ab1 with x_a?

Elapsed time in update: 1.871682
Epoch: 00 Iteration: 00001118/00100000 DLoss: 3.0236 Reg: 0.0102
L_total: 21.1056, L_gan: 3.6200, Lx: 2.4833, Lxp: 3.0208, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 6.0997, id: 3.1387, pid:2.7432, teacher: 0.0000
Elapsed time in update: 1.881756
Epoch: 00 Iteration: 00001119/00100000 DLoss: 2.9715 Reg: 0.0053
L_total: 19.6697, L_gan: 2.6888, Lx: 2.2801, Lxp: 2.8005, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 4.9748, id: 3.3738, pid:3.5517, teacher: 0.0000
Elapsed time in update: 1.872519
Epoch: 00 Iteration: 00001120/00100000 DLoss: 2.9412 Reg: 0.0083
L_total: 27.2018, L_gan: 3.5204, Lx: 2.7357, Lxp: 3.2405, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 6.1251, id: 5.6033, pid:5.9767, teacher: 0.0000
Elapsed time in update: 1.905725
Epoch: 00 Iteration: 00001121/00100000 DLoss: 2.9913 Reg: 0.0062
L_total: 23.0859, L_gan: 2.7672, Lx: 2.8575, Lxp: 3.2430, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 4.9441, id: 3.9912, pid:5.2831, teacher: 0.0000
Elapsed time in update: 1.884052
Epoch: 00 Iteration: 00001122/00100000 DLoss: 2.9047 Reg: 0.0091
L_total: 26.8263, L_gan: 3.5744, Lx: 2.8879, Lxp: 3.3251, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 6.6130, id: 5.2802, pid:5.1456, teacher: 0.0000
Elapsed time in update: 1.870903
Epoch: 00 Iteration: 00001123/00100000 DLoss: 2.9207 Reg: 0.0065
L_total: 23.1251, L_gan: 3.0145, Lx: 3.0704, Lxp: 3.3914, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 4.5945, id: 3.7942, pid:5.2601, teacher: 0.0000
Elapsed time in update: 2.011813
Epoch: 00 Iteration: 00001124/00100000 DLoss: 3.0592 Reg: 0.0076
L_total: 26.3440, L_gan: 3.4274, Lx: 2.3585, Lxp: 2.7218, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 6.1779, id: 6.2602, pid:5.3983, teacher: 0.0000
Elapsed time in update: 1.869946
Epoch: 00 Iteration: 00001125/00100000 DLoss: 2.9577 Reg: 0.0058
L_total: 26.9961, L_gan: 2.7294, Lx: 2.5079, Lxp: 2.7976, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 6.9536, id: 7.3599, pid:4.6478, teacher: 0.0000
Elapsed time in update: 1.882983
Epoch: 00 Iteration: 00001126/00100000 DLoss: 2.9359 Reg: 0.0096
L_total: 23.7432, L_gan: 3.7142, Lx: 2.4785, Lxp: 2.9254, Lrecycle:0.0000, Lf: 0.0000, Ls: 0.0000, Recon-id: 5.4275, id: 3.7460, pid:5.4517, teacher: 0.0000

and I have a picture of a dress and a set of pictures of people, can I change the clothes of the people to the given clothes through this project and how do I do that?

Hi,
Could you provide me with the trained model and training configs of the supervised learning for MSMT17 dataset?

Thanks for you sharing codes. I have a question, I can not find opt.alpha in trainer.py about predict_label function. opt seems not to be defined. Could you explain it?

Can you please give the setting for transfer learning in a more detailed manner?
How we have to use the code for transfer learning?

I want to test your weights on a video or image?How can I do it?

Hello, in your ft_netAB id encoder defined in reIDmodel.py you detach the id_embedding:

f = self.model.partpool(x)
f = f.view(f.size(0),f.size(1)*self.
f = f.detach() # no gradient 

In the gen_update loss function in trainer.py you calculate the id embedding reconstruction loss (as specified in your paper) but you are using the detached embeddings:

self.loss_gen_recon_f_a = self.recon_criterion(f_a_recon, f_a) if hyperparameters['recon_f_w'] > 0 else 0
self.loss_gen_recon_f_b = self.recon_criterion(f_b_recon, f_b) if hyperparameters['recon_f_w'] > 0 else 0 

Because the embeddings are detached these losses are not constraining the model, did I miss something ?

Hello, thank you very much for providing the implementation code of the DG-Net model.
I encountered some problems during the implementation of the project. I will be honored if you can give me some suggestions.

1. The difference of the model structure between the provided trained model and the code.
   I have successfully evaluated the DG-Net trained model that you provided in the github. However, when I resumed the training from the trained model that you provided, the project reported an error as follow:
   
   I found the extra weights were the non_local layer of the ResBlock. However, after I changed the res_type of the ContentEncoder and Decoder, the weights was still wrong.
   

2. The reapperance cannot achieve the expected map.
   I evaluated the DG-Net trained model that you provided and got the mAP for 0.8609 when alpha was 0.5. However, when I retrained, I only got the mAP for 0.8466. I had loaded the teacher model and the config was configs/latest.yaml. Is there anything else I did not notice?
   By the way, I used the reid_eval/test_2label.py for evaluate.

I will be grateful if you can give me some suggestions. Thank you！

we want to train the model on 4 GPUS:
python train.py --name latest --config configs/latest.yaml --gpu_ids 0,1,2,3
But the RuntimeError "CUDA out of memory" occurs, it seems only one GPU is used.
Any help?
Thank you.

hey，thank you for your code，i want to use your trained model to realize self-identity generation，how can i it？thank you

Hi Xiaodong Yang, Zhedong Zheng,

I am planning to use your model in one of our experimental project as base model for transfer learning.
And while studying I have noticed that your "val" (validation) dtat-set is subset of "train" (training) data-set. (Refer https://github.com/NVlabs/DG-Net/blob/master/prepare-market.py#L111)

And I believe that it is quite against my understanding.
So kindly explain why you have decided to have " validation data-set as subset of training data-set" ?

Hi, in DGNet_Trainer, you have gen_a, gen_b. I'm confused that in a single dataset training, do you need two generators a and b?
What is the meaning of feeding image a and image b in your trainer?
Thanks!

Thanks for your code. I have learned the paper and the code. I can't understand the role of 'alpha' in evaluate_gpu.py.
like this :
" alpha = [0, 0.5, -1]"
“if alpha[j] == -1:
qf[0:512] *= 0”
Would you please explain it for me？Why are u doing this?
Thank u.

Hi authors,
I have read your papers and have some question about architecture details. But I can't find supplementary material in this paper. Can you give me more detail or supplementary material
Thanks you.

I'm new to Re-ID,what should I do if I want to use the generated images to do data augmentation in other algorithms?And if I want to use show_smooth.py,how can I get samples in input_folder" visual_data"?Thank u a lot for your brilliant work. @layumi

I used the same hyperparameters setting for DukeMTMC and MSMT17 as the setting in Market1501 training config file. And I used the pretrained Resnet model from https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO as the teacher models.

Although I got a pretty nice result in Market1501, the result in the other two datasets are really poor(even both poorer than their teacher models).

Market1501:
torch.Size([3368, 1024])
Alpha:0.00 Rank@1:0.9374 Rank@5:0.9860 Rank@10:0.8490 mAP:0.8490
Alpha:0.50 Rank@1:0.9471 Rank@5:0.9872 Rank@10:0.8637 mAP:0.8637
Alpha:-1.00 Rank@1:0.9356 Rank@5:0.9837 Rank@10:0.8048 mAP:0.8048

Duke:
torch.Size([2228, 1024])
Alpha:0.00 Rank@1:0.6683 Rank@5:0.7742 Rank@10:0.8043 mAP:0.4390
Alpha:0.50 Rank@1:0.6849 Rank@5:0.7873 Rank@10:0.8187 mAP:0.4652
Alpha:-1.00 Rank@1:0.6688 Rank@5:0.7733 Rank@10:0.8021 mAP:0.4291

MSMT:
torch.Size([11659, 1024])
Alpha:0.00 Rank@1:0.4731 Rank@5:0.6029 Rank@10:0.6484 mAP:0.2736
Alpha:0.50 Rank@1:0.4754 Rank@5:0.6052 Rank@10:0.6544 mAP:0.2799
Alpha:-1.00 Rank@1:0.3923 Rank@5:0.5260 Rank@10:0.5779 mAP:0.1903

I also post the training loss log of DukeMTMC below.

• Does the teacher models' ability have a huge effect on the system's final ability?
• (小声哔哔：Could you share the hyperparameter training settings and teacher model on the other datasets?(Ｔ▽Ｔ)太穷了，跑一次要花50RMB，调不起参)
• I also wonder why you 大神们 know how to tune the hyperparameters, where I can learn some tricks about tuning? Only by practicing?

Your reply would be highly appreciated!

Hi, I am trying to understand what is the range of the score that comes out from the evaluate function

Putting the function here as reference

def evaluate(qf,ql,qc,gf,gl,gc):
    query = qf.view(-1,1)
    score = torch.mm(gf,query)
    score = score.squeeze(1).cpu()
    score = score.numpy()
    
    .....

I expected the scores to be in a range from 0 to 1, but it seems that the results here show values that can go negative and beyond 1. Is this expected behaviour? If yes, what is the score range as I would like to normalize these values.

Firstly, thanks for the work and sorry for my low knowledge.
i down the teach model and train the DGnet 100000 Times，but the source was so low ，has any met it and do you give me some suggest

pytorch/pytorch#15307

Also, Vgg16() is defined at https://github.com/NVlabs/DG-Net/blob/master/networks.py#L758 but is not imported into this file.

flake8 testing of https://github.com/NVlabs/DG-Net on Python 3.7.1

$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./utils.py:229:18: F821 undefined name 'load_lua'
        vgglua = load_lua(os.path.join(model_dir, 'vgg16.t7'))
                 ^
./utils.py:230:15: F821 undefined name 'Vgg16'
        vgg = Vgg16()
              ^
./utils.py:234:11: F821 undefined name 'Vgg16'
    vgg = Vgg16()
          ^
./networks.py:577:25: F821 undefined name 'SpectralNorm'
            self.conv = SpectralNorm(nn.Conv2d(input_dim, output_dim, kernel_size, stride, dilation=dilation, bias=self.use_bias))
                        ^
4     F821 undefined name 'SpectralNorm'
4

E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.

• F821: undefined name name
• F822: undefined name name in __all__
• F823: local variable name referenced before assignment
• E901: SyntaxError or IndentationError
• E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

what is the accuracy of the teacher model on market1501? if the teacher model will influence the final results of DG-net?

When i train the models on the MSMT17,the following error occurred:
Traceback (most recent call last):
File "test_2label.py", line 205, in
gallery_feature = extract_feature(model,dataloaders['gallery'])
File "test_2label.py", line 132, in extract_feature
x[0] = norm(x[0])
File "test_2label.py", line 110, in norm
fnorm = torch.norm(f, p=2, dim=1, keepdim=True)
File "/home/lei/.local/lib/python3.6/site-packages/torch/functional.py", line 705, in norm
return torch._C._VariableFunctions.norm(input, p, dim, keepdim=keepdim)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
How can i modify the preparation and evaluation code of the dataset MSMT17？Could you give me some suggwstions? Thank you !

I use pytorch1.0.0, but this module is not included, which version of torch do you use?

Hi，I installed apex. When I used apex:True, then appeared error about value cannot be converted to type float without overflow.
Error track：‘trainer.dis_update(images_a, images_b, config) -> scaled_loss.backward()’
Code environment：Pytho: 3.6.8, CUDA: 9.0, Pytorch: 1.0.1
I don't know if you have ever met，Could you help me? Thanks.

why did you use log-softmax in student-net, and softmax in teacher-net?

Firstly, thanks for the work and sorry for my low knowledge.

I want to give two specific image, source and target, and transfer pose of source to target. How can i do this, can you explain to me step by step?

Thanks.

Hi, when i use your model to start evaluation, i got this error as follow:
Traceback (most recent call last):
File "fid.py", line 23, in
import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
when i activated tensorflow env, and i run the code, i got this error as follow:
ModuleNotFoundError: No module named 'torch'
i don't know how to solve the problem.Can you give me some suggestions to solve the problem?Thank you.

Can you still provide duke, MSMT, CUHK03 data sets generated with DG-net?Thank you!

IndexError Traceback (most recent call last)
/visual_tools/test_folder.py in
59 # Setup model and data loader
60 if opts.trainer == 'DGNet':
---> 61 trainer = DGNet_Trainer(config)
62 else:
63 sys.exit("Only support DGNet")

/trainer.py in init(self, hyperparameters, gpu_ids)
289 #ID Loss
290 self.id_criterion = nn.CrossEntropyLoss()
--> 291 self.criterion_teacher = nn.KLDivLoss(size_average=False)
292 # Load VGG model if needed
293 if 'vgg_w' in hyperparameters.keys() and hyperparameters['vgg_w'] > 0:

/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/torch/nn/modules/loss.py in init(self, size_average, reduce, reduction)
361
362 def init(self, size_average=None, reduce=None, reduction='mean'):
--> 363 super(KLDivLoss, self).init(size_average, reduce, reduction)
364
365 @weak_script_method

/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/torch/nn/modules/loss.py in init(self, size_average, reduce, reduction)
14 super(_Loss, self).init()
15 if size_average is not None or reduce is not None:
---> 16 self.reduction = _Reduction.legacy_get_string(size_average, reduce)
17 else:
18 self.reduction = reduction

/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/torch/nn/_reduction.py in legacy_get_string(size_average, reduce, emit_warning)
47 ret = 'none'
48 if emit_warning:
---> 49 warnings.warn(warning.format(reduction='sum'))
50 return ret
51

IndexError: tuple index out of range

hi guys, thanks for your greate contributions. I am just curious while you do the supervised learning for MSMT17 datasets, did you reisze the images to the same size?(since the shape of the images vary in that dataset). If not, how did you deal with that?

According to this code, a pre-trained teacher model should be provided. I wonder how the teacher model was trained and its accuracy. Thanks！

Hello, can you tell me if you have only implemented the change of appearance space, but not the transformation of structure space. For example, to generate a visual Angle image that does not exist in the structure space. Thank u