sunarker / collaborative-learning-for-weakly-supervised-object-detection Goto Github PK

Hi, thanks for the sharing work.
I trained the model with
./experiments/scripts/train.sh 0 pascal_voc vgg16 wsddnpath
and I found in vgg16.yml it needs selective search. I tried to modified it into gt and I got

  File "./tools/trainval_net.py", line 114, in <module>
    imdb, roidb = combined_roidb(args.imdb_name)
  File "./tools/trainval_net.py", line 80, in combined_roidb
    roidbs = [get_roidb(s) for s in imdb_names.split('+')]
  File "./tools/trainval_net.py", line 77, in get_roidb
    roidb = get_training_roidb(imdb)
  File "/home/Collaborative-Learning-for-Weakly-Supervised-Object-Detection/tools/../lib/model/train_val.py", line 320, in get_training_roidb
    imdb.append_flipped_images()
  File "/home/Collaborative-Learning-for-Weakly-Supervised-Object-Detection/tools/../lib/datasets/imdb.py", line 126, in append_flipped_images
    boxes = self.roidb[i]['boxes'].copy()
KeyError: 'boxes'

the roidb object only has label key and no other key. I 'm confused about the data processing.
Can you give me some suggestions, thank you

Hi，thank you for your hard and wonderful work.
I want to konw which pytorch version do you use?
Thank you!Best wishes!

thank you very much!

Hi，I'd like to ask you a question.
What GPU does this program run on and how long it takes to train?
Best wishes！

Traceback (most recent call last):
File "./tools/trainval_net.py", line 150, in
max_iters=args.max_iters)
File "/home/nieqinqin/liuxiaoyu/CLWS/tools/../lib/model/train_val.py", line 365, in train_net
sw.train_model(max_iters)
File "/home/nieqinqin/liuxiaoyu/CLWS/tools/../lib/model/train_val.py", line 239, in train_model
lr, last_snapshot_iter, stepsizes, np_paths, ss_paths = self.initialize()
File "/home/nieqinqin/liuxiaoyu/CLWS/tools/../lib/model/train_val.py", line 179, in initialize
self.net.load_state_dict(model_dict)
File "/home/nieqinqin/liuxiaoyu/CLWS/tools/../lib/nets/network.py", line 609, in load_state_dict
nn.Module.load_state_dict(self, {k: state_dict[k] for k in list(self.state_dict())})
File "/home/nieqinqin/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in load_state_dict
.format(name, own_state[name].size(), param.size()))
RuntimeError: While copying the parameter named cls_score_net.weight, whose dimensions in the model are torch.Size([20, 2048]) and whose dimensions in the checkpoint are torch.Size([20, 4096]).
Command exited with non-zero status 1
7.33user 5.80system 0:13.57elapsed 96%CPU (0avgtext+0avgdata 2171288maxresident)k
0inputs+8outputs (0major+709527minor)pagefaults 0swaps

Hello, thanks for the sharing work.
I trained the model with
./experiments/scripts/train.sh 0 pascal_voc vgg16 path_to_wsddn_pretrained_model
But I got this questions：

(torch0.4) Alex@oo:/home/sdb1/csj/idiea/Collaborative-Learning$ bash ./experiments/scripts/train.sh 0 pascal_voc vgg16 path_to_wsddn_pretrained_model

• set -e
• export PYTHONUNBUFFERED=True
• PYTHONUNBUFFERED=True
• unset PYTHONPATH
• GPU_ID=0
• DATASET=pascal_voc
• NET=vgg16
• WSDDN=path_to_wsddn_pretrained_model
  ./experiments/scripts/train.sh: 行 12: 未预期的符号 '(' 附近有语法错误 (torch0.4) Alex@oo:/home/sdb1/csj/idiea/Collaborative-Learning$

I've tried a lot of methods but i can't work.
Could you give me some suggestions?

cross_entropy: 0.001210
lr: 0.001000
speed: 2.425s / iter
/gruntdata/disk2/hm/CLWSOD/tools/../lib/nets/network.py:569: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
cross_entropy, total_loss = self._losses['wsddn_loss'].data[0],
/gruntdata/disk2/hm/CLWSOD/tools/../lib/nets/network.py:570: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
self._losses['total_loss'].data[0]
iter: 2 / 200000, total loss: 1.993461
cross_entropy: 0.000035
lr: 0.001000
speed: 1.868s / iter
iter: 3 / 200000, total loss: 0.652638
cross_entropy: 0.016665
lr: 0.001000
speed: 1.693s / iter
iter: 4 / 200000, total loss: 0.393473
cross_entropy: 0.001328
lr: 0.001000
speed: 1.708s / iter
iter: 5 / 200000, total loss: 0.351719
cross_entropy: 0.000444
lr: 0.001000
speed: 1.620s / iter
iter: 6 / 200000, total loss: 0.543326
cross_entropy: 0.050670
lr: 0.001000
speed: 1.600s / iter
iter: 7 / 200000, total loss: 0.261543
cross_entropy: 0.000296
lr: 0.001000
speed: 1.564s / iter
iter: 8 / 200000, total loss: 0.783304
cross_entropy: 0.024824
lr: 0.001000
speed: 1.529s / iter
iter: 9 / 200000, total loss: 0.537496
cross_entropy: 0.011513
lr: 0.001000
speed: 1.510s / iter
iter: 10 / 200000, total loss: 0.964071
cross_entropy: 0.010650
lr: 0.001000
speed: 1.489s / iter
iter: 11 / 200000, total loss: 0.296966
cross_entropy: 0.020692
lr: 0.001000
speed: 1.472s / iter
iter: 12 / 200000, total loss: 0.546587
cross_entropy: 0.044390
lr: 0.001000
speed: 1.480s / iter
iter: 13 / 200000, total loss: 0.693391
cross_entropy: 0.001768
lr: 0.001000
speed: 1.479s / iter
iter: 14 / 200000, total loss: 0.190509
cross_entropy: 0.051802
lr: 0.001000
speed: 1.474s / iter
iter: 15 / 200000, total loss: 0.302866
cross_entropy: 0.053017
lr: 0.001000
speed: 1.476s / iter
iter: 16 / 200000, total loss: 0.468978
cross_entropy: 0.000957
lr: 0.001000
speed: 1.456s / iter
iter: 17 / 200000, total loss: 0.609222
cross_entropy: 0.007434
lr: 0.001000
speed: 1.457s / iter
iter: 18 / 200000, total loss: 0.089435
cross_entropy: 0.003355
lr: 0.001000
speed: 1.458s / iter
iter: 19 / 200000, total loss: 0.506788
cross_entropy: 0.002159
lr: 0.001000
speed: 1.464s / iter
iter: 20 / 200000, total loss: 0.507251
cross_entropy: 0.020046
lr: 0.001000
speed: 1.464s / iter
iter: 21 / 200000, total loss: 0.365586
cross_entropy: 0.113681
lr: 0.001000
speed: 1.455s / iter
iter: 22 / 200000, total loss: 0.184315
cross_entropy: 0.084765
lr: 0.001000
speed: 1.467s / iter
iter: 23 / 200000, total loss: 0.200998
cross_entropy: 0.048887
lr: 0.001000
speed: 1.458s / iter
iter: 24 / 200000, total loss: 0.124370
cross_entropy: 0.003205
lr: 0.001000
speed: 1.461s / iter
iter: 25 / 200000, total loss: 0.102922
cross_entropy: 0.059250
lr: 0.001000
speed: 1.467s / iter
iter: 26 / 200000, total loss: 0.175924
cross_entropy: 0.031119
lr: 0.001000
speed: 1.495s / iter
iter: 27 / 200000, total loss: 0.185290
cross_entropy: 0.002968
lr: 0.001000
speed: 1.493s / iter
iter: 28 / 200000, total loss: 0.163398
cross_entropy: 0.005777
lr: 0.001000
speed: 1.484s / iter
Traceback (most recent call last):
File "./tools/trainval_net.py", line 149, in
max_iters=args.max_iters)
File "/gruntdata/disk2/hm/CLWSOD/tools/../lib/model/train_val.py", line 380, in train_net
sw.train_model(max_iters)
File "/gruntdata/disk2/hm/CLWSOD/tools/../lib/model/train_val.py", line 294, in train_model
self.net.train_step(blobs, self.optimizer)
File "/gruntdata/disk2/hm/CLWSOD/tools/../lib/nets/network.py", line 573, in train_step
self._losses['total_loss'].backward()
File "/gruntdata/disk1/anaconda3/envs/hm3/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/gruntdata/disk1/anaconda3/envs/hm3/lib/python3.6/site-packages/torch/autograd/init.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: CUDA error: out of memory
Command exited with non-zero status 1
339.25user 70.95system 1:04.60elapsed 634%CPU (0avgtext+0avgdata 3661156maxresident)k
0inputs+184outputs (0major+2307976minor)pagefaults 0swaps

Hello,Thanks for sharing your great works again.
Recently, when I changed the Pascal VOC2007 training set to my own training set (only 6 classes of pictures), I encountered a problem that the class_score_net.bias dimension did not match.
Can you share how the WSDDN pre-training model was obtained?
Thank you very much!

We have read the source code and found there are some details that are not same as the published paper:

1. In the code, we find the proposals after NMS within top 50 scores are used as the pseudo ground truths for Faster-RCNN,

def generate_pseudo_gtbox(boxes, cls_prob, im_labels):
    """Get proposals from fuse_matrix
    inputs are all variables"""
    pre_nms_topN = 50
    nms_Thresh = 0.1
    
    num_images, num_classes = im_labels.size()
    boxes = boxes[:,1:]
    assert num_images == 1, 'batch size shoud be equal to 1'
    im_labels_tmp = im_labels[0, :]
    labelList = im_labels_tmp.data.nonzero().view(-1)
    
    gt_boxes = []
    gt_classes = []
    gt_scores = []
    
    for i in labelList:
        scores, order = cls_prob[:,i].contiguous().view(-1).sort(descending=True)
        if pre_nms_topN > 0:
          order = order[:pre_nms_topN]
          scores = scores[:pre_nms_topN].view(-1, 1)
          proposals = boxes[order.data, :]
          
        keep = nms(torch.cat((proposals, scores), 1).data, nms_Thresh)
        proposals = proposals[keep, :]
        scores = scores[keep,]
        gt_boxes.append(proposals)
        gt_classes.append(torch.ones(keep.size(0),1)*(i+1))  # return idx=class+1 to include the background
        gt_scores.append(scores.view(-1,1))
            
    gt_boxes = torch.cat(gt_boxes)
    gt_classes = torch.cat(gt_classes)
    gt_scores = torch.cat(gt_scores)
    proposals = {'gt_boxes' : gt_boxes,
                 'gt_classes': gt_classes,
                 'gt_scores': gt_scores}

while the paper uses the top scoring box.

Max-out Strategy The predictions of DS and DW could be inaccurate, especially in the initial rounds of training. For measuring the prediction consistency, it is important to select only the most confident predictions. We thus apply a Max-out strategy to filter out most predictions. For each positive category, only the region with highest prediction score by DW is chosen.

2. Also, the bootstrap_cross_entropy method is different with that in the paper when the cfg.TRAIN.ISHARD is set True, which is the case when we run the code without any modification.

if ishard:
        _,idx = input_prob.max(1)
        target_onehot = target_onehot * beta + \
                    Variable(input.data.new(input.data.size()).zero_()).scatter_(1, idx.view(-1,1), 1) * (1-beta)
    else:
        target_onehot = target_onehot * beta + input_prob * (1-beta)

We have run the code and get mAP 47.0 on VOC2007 test dataset, inferior than the mAP 48.2 posted in the paper. We wonder if the reason is the 2 differences.
Will you please explain the 2 differences and tell us how to reproduce similar results as in the paper?
Thanks!

Hi, thanks for your excellent work!
But I have some questions during testing and training,and hoping for your help. As I read your paper, pretrained wsddn model that downloaded from here should generate mAP 28.5 on voc2007 dataset, but I just got 25.7, and I have no idea what's wrong. Similarly, I used this pretrained model to train:
./experiments/scripts/train.sh 0 pascal_voc vgg16 path_to_wsddn_pretrained_model, after training 70000 iters, the result was just 45.8, vs your result 48.3
My environment setting:
pytorch 0.2
cuda 9.0
python 3.6
I am not sure if the selective search data affects final results cause you didn't give your pre-computed proposals. Usually there are about 2K proposals for an image. So could you please give me some advice and share your pre-computed proposals? Thanks very much.

Thank you for sharing your wonderful work.I am a beginner of weak supervised target detection.
I encountered the following problems.
(W) mickey@a208-System-Product-Name:~/project/Collaborative-Learning-for-Weakly-Supervised-Object-Detection/experiments/scripts$ sh train.sh 0 pascal_voc vgg16 /home/mickey/project/Collaborative-Learning-for-Weakly-Supervised-Object-Detection/data/imagenet_weights/voc07_wsddn_pre.pth/

• set -e
• export PYTHONUNBUFFERED=True
• PYTHONUNBUFFERED=True
• unset PYTHONPATH
• GPU_ID=0
• DATASET=pascal_voc
• NET=vgg16
• WSDDN=/home/mickey/project/Collaborative-Learning-for-Weakly-Supervised-Object-Detection/data/imagenet_weights/voc07_wsddn_pre.pth/
• array=($@)
• len=4
• EXTRA_ARGS=
• EXTRA_ARGS_SLUG=
• case ${DATASET} in
• TRAIN_IMDB=voc_2007_trainval
• TEST_IMDB=voc_2007_test
• STEPSIZE='[50000]'
• ITERS=200000
• ANCHORS='[8,16,32]'
• RATIOS='[0.5,1,2]'
  ++ date +%Y-%m-%d_%H-%M-%S
• LOG=experiments/logs/vgg16_voc_2007_trainval__vgg16.txt.2019-04-14_21-42-21
  train.sh: line 49: syntax error near unexpected token `>'

Excuse me, have you tried using fast_rcnn without the RPN module. I have tried, but the result is not ideal. Is it because the RPN module can generate better proposals? I want to know why, if you know. Thank you!

Set proposal method: gt
Appending horizontally-flipped training examples...
wrote gt roidb to /home/yang/Documents/Collaborative-Learning-for-Weakly-Supervised-Object-Detection-master/data/cache/voc_2007_trainval_gt_roidb.pkl
Traceback (most recent call last):
File "./tools/trainval_net.py", line 114, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/trainval_net.py", line 80, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/trainval_net.py", line 80, in
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/trainval_net.py", line 77, in get_roidb
roidb = get_training_roidb(imdb)
File "/home/yang/Documents/Collaborative-Learning-for-Weakly-Supervised-Object-Detection-master/tools/../lib/model/train_val.py", line 319, in get_training_roidb
imdb.append_flipped_images()
File "/home/yang/Documents/Collaborative-Learning-for-Weakly-Supervised-Object-Detection-master/tools/../lib/datasets/imdb.py", line 125, in append_flipped_images
boxes = self.roidb[i]['boxes'].copy()
KeyError: 'boxes'
Command exited with non-zero status 1
2.04user 2.67system 0:01.48elapsed 318%CPU (0avgtext+0avgdata 190308maxresident)k
0inputs+1128outputs (0major+33861minor)pagefaults 0swaps

Hello,Thanks for sharing your great works!
But when I trained the model with ./experiments/scripts/train.sh 0 pascal_voc vgg16 voc12_wsddn_pre, I encountered the error

Traceback (most recent call last):
  File "./tools/trainval_net.py", line 149, in <module>
    max_iters=args.max_iters)
  File "/home/amax/GWL/pytorch-faster-rcnn/tools/../lib/model/train_val.py", line 365, in train_net
    sw.train_model(max_iters)
  File "/home/amax/GWL/pytorch-faster-rcnn/tools/../lib/model/train_val.py", line 283, in train_model
    self.net.train_step(blobs, self.optimizer)
  File "/home/amax/GWL/pytorch-faster-rcnn/tools/../lib/nets/network.py", line 574, in train_step
    self._losses['total_loss'].backward()
  File "/home/amax/.local/lib/python2.7/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/amax/.local/lib/python2.7/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=False)  # allow_unreachable flag
RuntimeError: CUDA error: out of memory
Command exited with non-zero status 1
13.82user 11.67system 0:13.76elapsed 185%CPU (0avgtext+0avgdata 3269864maxresident)k
464inputs+24outputs (0major+1764047minor)pagefaults 0swaps

My PC's details are as follows:

CPU：Intel® Xeon(R) CPU E5-1650 v4 @ 3.60GHz × 12 
GPU：GeForce GTX 1080 Ti/PCIe/SSE2
Memory: 32G
Disk：228G
Ubuntu：16.04LTS
Cuda：9.0.176
Cudnn：7.4.1
torch：0.4.1

Can you give me some advice?
Thank you

Thanks for sharing your great work!

I'm currently working on a project about object detection refinement. So could you share me your predicted result files on VOC 07/12 trainval/test set? So that I could do some experiments on it and cite your work in the future?

Looking forward to your response!

when i run
./experiments/scripts/test.sh 1 pascal_voc vgg16 voc07_wsddn_pre
it returns :

Command terminated by signal 11 1.81user 3.24system 0:01.81elapsed 278%CPU (0avgtext+0avgdata 137680maxresident)k 1280inputs+0outputs (0major+20019minor)pagefaults 0swaps

and system stops.

when I tried demo.py
and I get this :

Segmentation fault (core dumped)

how can i fixed it.
pleas help thank you

I'm using cuda7.5 / torch-0.2.0 / Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-33-generic x86_64)
GPU: K80

when i see the classes，there is 20 classes，removed the background class。so，i don't know why remove the background from the category list??how can i understand it?please.