Code Monkey home page Code Monkey logo

detectron-da-faster-rcnn's Introduction

Domain Adaptive Faster R-CNN in Detectron

Updates

Introduction

This is a Caffe2 implementation of 'Domain Adaptive Faster R-CNN for Object Detection in the Wild', implemented by Haoran Wang([email protected]). The original paper can be found here. This implementation is built on Detectron @ 5ed75f9.

If you find this repository useful, please cite the oringinal paper:

@inproceedings{chen2018domain,
  title={Domain Adaptive Faster R-CNN for Object Detection in the Wild},
      author =     {Chen, Yuhua and Li, Wen and Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
      booktitle =  {Computer Vision and Pattern Recognition (CVPR)},
      year =       {2018}
  }

and Detectron:

@misc{Detectron2018,
  author =       {Ross Girshick and Ilija Radosavovic and Georgia Gkioxari and
                  Piotr Doll\'{a}r and Kaiming He},
  title =        {Detectron},
  howpublished = {\url{https://github.com/facebookresearch/detectron}},
  year =         {2018}
}

Installation

Please follow the instruction in Detectron to install and use Detectron-DomainAdaptive-Faster-RCNN.

Usage Example

An example of adapting from Sim10k dataset to Cityscapes dataset is provided:

  1. Download the Cityscapes datasets from here and Sim10k datasets from here.

  2. Convert the labels of Cityscapes datasets and Sim10k datasets to coco format using the scripts 'tools/convert_cityscapes_to_caronly_coco.py' and 'tools/convert_sim10k_to_coco.py'.

  3. Convert ImageNet-pretrained VGG16 Caffe model to Detectron format with 'tools/pickle_caffe_blobs.py' or use my converted VGG16 model in here

  4. Train the Domain Adaptive Faster R-CNN:

    cd $DETECTRON
    python2 tools/train_net.py --cfg configs/da_faster_rcnn_baselines/e2e_da_faster_rcnn_vgg16-sim10k.yaml
    
  5. Test the trained model:

    cd $DETECTRON
    python2 tools/test_net.py --cfg configs/da_faster_rcnn_baselines/e2e_da_faster_rcnn_vgg16-sim10k.yaml TEST.WEIGHTS /<path_to_trained_model>/model_final.pkl NUM_GPUS 1
    

Pretrained Model & Results

The best results for different adaptation are reported. Due to the instable nature of adversarial training, the best models are obtained through a model selection on a randomly picked mini validation set.

image instsnace consistency car AP Pretrained models
Faster R-CNN 32.58
DA Faster R-CNN 38.60 model
DA Faster R-CNN 35.55 model
DA Faster R-CNN 39.23 model
DA Faster R-CNN 40.01 model

Other Implementation

da-faster-rcnn based on Caffe. (original code by paper authors)

Domain-Adaptive-Faster-RCNN-PyTorch based on PyTorch and maskrcnn-benchmark.

sa-da-faster based on PyTorch and maskrcnn-benchmark.

detectron-da-faster-rcnn's People

Contributors

krumo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

detectron-da-faster-rcnn's Issues

Target set labels are used during training

Hi @krumo,

I've been trying to get the original code in caffe 1.0 by @yuhuayc working, and now I am switching to your code because it is better and has DA consistency loss code. I am trying to train a domain adaptive detector using an unlabeled target set. This has posed some problems because an empty bounding box list in the labels is not accepted by the code. By adding dummy labeling (a single arbitrary bbx) all problems were resolved in the caffe 1.0 implementation, but in the current implementation I came to some extra problems.

The main issue is that the training process in your implementation does not seem to be independant of the target set labels. They are used by the GenerateProposalLabels python layer for making a selection of 256 out of 2000 rpn proposals. I would regard this as cheating, because this makes sure that the L_ins is focussed on the most relevant rois, selected with respect to the ground truth labels.

In the original implementation by @yuhuayc this is avoided by feeding, besides the selection of 256(128) rois, all 2000 rpn proposals into the RoiPooling down to the bottleneck features, to the L_ins. This is then done for both source and target images. (https://github.com/yuhuayc/da-faster-rcnn/blob/master/models/da_faster_rcnn/train.prototxt here 'concat_rois' is defined as input for roi pooling, and after the bottleneck features they are split again in the 'slice_feats' layer.)

(This might also explain why you have changed lambda to 0.2; there are roughly 8 times less instances in the sum of L_ins.)

I've trained a network with source: coco_2017_train + target:unlabeled_imgs(with arbitrary bbxes) and validated it on coco_2017_val. The overall AP of 0.141 seems quite low. It could be because of my unlabeled image set.

Also, do you perhaps know what code is used to produce the original results (inc L_cst) in the paper? @yuhuayc is reffering to your repo for that. Is it?

error while training without Domain Adaptation on custom datasets

Hi,

I was trying to compare a Domain Adaptive (DA) model against a normally (i.e. without any adaptation) trained model on my custom datasets. DA model is successfully trained but I am getting following error while trying to train the normal model (I set DOMAIN_ADAPTATION: False and passed DATASETS: my_dataset_path to do normal training). While I am trying to dig deeper, could you please provide any pointer as to what may be wrong? Thanks.
Note that I am able to train normal models using detectron's original repo.

##################### Error log #############

[E pybind_state.h:424] Exception encountered running PythonOp function: KeyError: ('da_rois',)

At:
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/roi_data/fast_rcnn.py(126): add_fast_rcnn_blobs
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/ops/generate_proposal_labels.py(52): forward

[E net_async_base.cc:377] [enforce fail at pybind_state.h:425] . Exception encountered running PythonOp function: KeyError: ('da_rois',)

At:
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/roi_data/fast_rcnn.py(126): add_fast_rcnn_blobs
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/ops/generate_proposal_labels.py(52): forward

Error from operator:
input: "gpu_0/rpn_rois" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois" output: "gpu_0/labels_int32" output: "gpu_0/bbox_targets" output: "gpu_0/bbox_inside_weights" output: "gpu_0/bbox_outside_weights" name: "GenerateProposalLabelsOp:rpn_rois,roidb,im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:1" } arg { name: "grad_output_indices" } device_option { device_type: 0 }Error from operator:
input: "gpu_0/rpn_rois" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois" output: "gpu_0/labels_int32" output: "gpu_0/bbox_targets" output: "gpu_0/bbox_inside_weights" output: "gpu_0/bbox_outside_weights" name: "GenerateProposalLabelsOp:rpn_rois,roidb,im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:1" } arg { name: "grad_output_indices" } device_option { device_type: 1 device_id: 0 }frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x50 (0x7f3b8b5a8000 in /home/neo/ML/pytorch/build/lib/libc10.so)
frame #1: + 0x9ddfa (0x7f3b8bce1dfa in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #2: + 0x9b98b (0x7f3b8bcdf98b in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #3: + 0xea2b3 (0x7f3b8bd2e2b3 in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #4: + 0xe8f74 (0x7f3b8bd2cf74 in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #5: caffe2::AsyncNetBase::run(int, int) + 0x144 (0x7f3b6bf88eb4 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #6: + 0x159a0a9 (0x7f3b6bf8f0a9 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #7: c10::ThreadPool::main_loop(unsigned long) + 0x283 (0x7f3b6b08f183 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #8: + 0x91c10 (0x7f3b916a2c10 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
frame #9: + 0x8184 (0x7f3b981dd184 in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #10: clone + 0x6d (0x7f3b97f0a37d in /lib/x86_64-linux-gnu/libc.so.6)
, op Python
[E net_async_base.cc:129] Rethrowing exception from the run of 'generalized_rcnn'
WARNING workspace.py: 205: Original python traceback for operator 38 in network generalized_rcnn in exception above (most recent call last):
WARNING workspace.py: 210: File "tools/train_net.py", line 132, in
WARNING workspace.py: 210: File "tools/train_net.py", line 114, in main
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/utils/train.py", line 53, in train_model
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/utils/train.py", line 145, in create_model
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/model_builder.py", line 125, in create
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/model_builder.py", line 90, in generalized_rcnn
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/model_builder.py", line 241, in build_generic_detection_model
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/optimizer.py", line 40, in build_data_parallel_model
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/optimizer.py", line 63, in _build_forward_graph
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/model_builder.py", line 193, in _single_gpu_build_func
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/rpn_heads.py", line 51, in add_generic_rpn_outputs
WARNING workspace.py: 210: File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/modeling/rpn_heads.py", line 160, in add_single_scale_rpn_losses
Traceback (most recent call last):
File "tools/train_net.py", line 132, in
main()
File "tools/train_net.py", line 114, in main
checkpoints = detectron.utils.train.train_model()
File "/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/utils/train.py", line 67, in train_model
workspace.RunNet(model.net.Proto().name)
File "/home/neo/ML/pytorch/build/caffe2/python/workspace.py", line 237, in RunNet
StringifyNetName(name), num_iter, allow_fail,
File "/home/neo/ML/pytorch/build/caffe2/python/workspace.py", line 198, in CallWithExceptionIntercept
return func(*args, **kwargs)
RuntimeError: [enforce fail at pybind_state.h:425] . Exception encountered running PythonOp function: KeyError: ('da_rois',)

At:
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/roi_data/fast_rcnn.py(126): add_fast_rcnn_blobs
/home/neo/ML/Detectron-DA-Faster-RCNN/detectron/ops/generate_proposal_labels.py(52): forward

Error from operator:
input: "gpu_0/rpn_rois" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois" output: "gpu_0/labels_int32" output: "gpu_0/bbox_targets" output: "gpu_0/bbox_inside_weights" output: "gpu_0/bbox_outside_weights" name: "GenerateProposalLabelsOp:rpn_rois,roidb,im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:1" } arg { name: "grad_output_indices" } device_option { device_type: 0 }Error from operator:
input: "gpu_0/rpn_rois" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois" output: "gpu_0/labels_int32" output: "gpu_0/bbox_targets" output: "gpu_0/bbox_inside_weights" output: "gpu_0/bbox_outside_weights" name: "GenerateProposalLabelsOp:rpn_rois,roidb,im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:1" } arg { name: "grad_output_indices" } device_option { device_type: 1 device_id: 0 }frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x50 (0x7f3b8b5a8000 in /home/neo/ML/pytorch/build/lib/libc10.so)
frame #1: + 0x9ddfa (0x7f3b8bce1dfa in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #2: + 0x9b98b (0x7f3b8bcdf98b in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #3: + 0xea2b3 (0x7f3b8bd2e2b3 in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #4: + 0xe8f74 (0x7f3b8bd2cf74 in /home/neo/ML/pytorch/build/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #5: caffe2::AsyncNetBase::run(int, int) + 0x144 (0x7f3b6bf88eb4 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #6: + 0x159a0a9 (0x7f3b6bf8f0a9 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #7: c10::ThreadPool::main_loop(unsigned long) + 0x283 (0x7f3b6b08f183 in /home/neo/ML/pytorch/build/lib/libcaffe2.so)
frame #8: + 0x91c10 (0x7f3b916a2c10 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
frame #9: + 0x8184 (0x7f3b981dd184 in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #10: clone + 0x6d (0x7f3b97f0a37d in /lib/x86_64-linux-gnu/libc.so.6)

Few questions for your hyperparameters

Hello, I'm trying to reproduce your work but have several issues.
(I'm new to CAFFE2)

Q1) Did you set the same hyperparameter for image level, image + instance level, image + instance + consistency loss setting each?

Q2) I can't not find your configuration about weight loss. On the original paper, weight(lambda) for image level, instance level and consistency loss are set to 0.1. I checked "DA_IMG_GRL_WEIGHT" and "DA_INS_GRL_WEIGHT", but it seems not the same one with lambda. Where can I find one or did you just set it to 1.0?

Q3) On implementation from Caffe (https://github.com/yuhuayc/da-faster-rcnn) , they set lr_mult 10 times higher for instance level domain classifiers, did you set any hyperparameters reponsible for lr_mult?

Get error when training with custom dataset

Hello, I try to train DA-Faster RCNN with my custom dataset using the VGG16 model but got some errors when it train, How can I fix it ?

...
...
[E net_async_base.cc:145] Rethrowing exception from the run of 'generalized_rcnn'
WARNING workspace.py: 234: Original python traceback for operator `143` in network `generalized_rcnn` in exception above (most recent call last):
WARNING workspace.py: 239:   File "tools/train_net.py", line 132, in <module>
WARNING workspace.py: 239:   File "tools/train_net.py", line 114, in main
WARNING workspace.py: 239:   File "/workingspace/detectron/utils/train.py", line 53, in train_model
WARNING workspace.py: 239:   File "/workingspace/detectron/utils/train.py", line 145, in create_model
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/model_builder.py", line 125, in create
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/model_builder.py", line 90, in generalized_rcnn
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/model_builder.py", line 241, in build_generic_detection_model
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/optimizer.py", line 40, in build_data_parallel_model
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/optimizer.py", line 63, in _build_forward_graph
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/model_builder.py", line 193, in _single_gpu_build_func
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/rpn_heads.py", line 51, in add_generic_rpn_outputs
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/rpn_heads.py", line 133, in add_single_scale_rpn_losses
WARNING workspace.py: 239:   File "/workingspace/detectron/modeling/detector.py", line 407, in MaskingInput
Traceback (most recent call last):
  File "tools/train_net.py", line 132, in <module>
    main()
  File "tools/train_net.py", line 114, in main
    checkpoints = detectron.utils.train.train_model()
  File "/workingspace/detectron/utils/train.py", line 67, in train_model
    workspace.RunNet(model.net.Proto().name)
  File "/usr/local/lib/python3.6/dist-packages/caffe2/python/workspace.py", line 266, in RunNet
    StringifyNetName(name), num_iter, allow_fail,
  File "/usr/local/lib/python3.6/dist-packages/caffe2/python/workspace.py", line 227, in CallWithExceptionIntercept
    return func(*args, **kwargs)
  File "/workingspace/detectron/ops/generate_proposal_labels.py", line 52, in forward
    fast_rcnn_roi_data.add_fast_rcnn_blobs(blobs, im_scales, roidb)
  File "/workingspace/detectron/roi_data/fast_rcnn.py", line 121, in add_fast_rcnn_blobs
    frcn_blobs = _sample_rois(entry, im_scales[im_i], im_i)
  File "/workingspace/detectron/roi_data/fast_rcnn.py", line 186, in _sample_rois
    roidb['bbox_targets'][keep_inds, :]
  File "/workingspace/detectron/roi_data/fast_rcnn.py", line 291, in _expand_bbox_targets
    bbox_targets[ind, start:end] = bbox_target_data[ind, 1:]
ValueError: could not broadcast input array from shape (4) into shape (0)

libavcodec-ffmpeg.so.56: cannot open shared object file: No such file or directory

Running
python2 tools/train_net.py --cfg configs/da_faster_rcnn_baselines/e2e_da_faster_rcnn_vgg16-sim10k.yaml

Traceback (most recent call last):
  File "tools/train_net.py", line 26, in <module>
    import cv2  # NOQA (Must import before importing caffe2 due to bug in cv2)
ImportError: libavcodec-ffmpeg.so.56: cannot open shared object file: No such file or directory

Running ffmpeg -v on the terminal, I get

  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100

Running
python tools/train_net.py --cfg configs/da_faster_rcnn_baselines/e2e_da_faster_rcnn_vgg16-sim10k.yaml throws this error.

Traceback (most recent call last):
  File "tools/train_net.py", line 38, in <module>
    from detectron.core.test_engine import run_inference
  File "/home/barock/detectron/detectron/core/test_engine.py", line 34, in <module>
    from detectron.core.rpn_generator import generate_rpn_on_dataset
  File "/home/barock/detectron/detectron/core/rpn_generator.py", line 41, in <module>
    from detectron.datasets import task_evaluation
  File "/home/barock/detectron/detectron/datasets/task_evaluation.py", line 47, in <module>
    import detectron.datasets.json_dataset_evaluator as json_dataset_evaluator
  File "/home/barock/detectron/detectron/datasets/json_dataset_evaluator.py", line 34, in <module>
    import detectron.utils.boxes as box_utils
  File "/home/barock/detectron/detectron/utils/boxes.py", line 51, in <module>
    import detectron.utils.cython_bbox as cython_bbox
ImportError: /home/barock/detectron/detectron/utils/cython_bbox.so: undefined symbol: _Py_ZeroStruct

Kindly help me in resolving the error.

mAP or mAP50

image
I found that it was written like this in the article. Is the evaluation index used in your experiment results AP (0.5:0.95) or AP50 (iou=0.5)? The term ‘mAP’ used in the article is a bit confusing.
Thanks!

Few questions for reproducing the results

Hi, I'm trying to reproduce your work for Sim10k->Cityscapes dataset but have several issues.
Q1: loss_da and loss_dc did not converge(loss_da=0.69,loss_dc=0.68) during the 70000 iterations training process. Is that the same for you? Did different optimization objectives between L_det and L_img+L_ins+L_cst lead to this result?

Q2: I reproduced 34.9 AP by using the image, instance, and consistency loss for Sim10k->Cityscapes task and config file e2e_da_faster_rcnn_vgg16-sim10k.yaml. Actually, I observed unstable results in different training sessions. Is it related to the optimization objective mentioned in Q1?

Reproduce the results of the baseline in SIM 10k to cityscape

I am planning to reproduce the results of the baseline in SIM 10k to cityscape on Pytorch, but I got a very high result of about 42%. And dafaster got a normal result of about 40%.

In fact, I have similar results from KITTI to cityscape. I may not be right, but I can't find the reason. Is there any detail to be aware of when implementing the baseline? Thank you!

Sim10k dataset

The link for the sim10k dataset seems to be unavailable. Do you have a copy or any other URL? Thanks.

Reproduce results (image/instance/consistency)

Hi! Nice work. I wanted to ask how to reproduce the results that you have on README:
image | instance | consistency
How do you activate/deactivate each of the respective setups (w/ image/instance/consistency)?
I have been trying several things, including changing the weights on configs but I am not sure whether this is how you do it. Also how to deal with the consistency.
Thanks!!!!

Qestions for Adaptation Strategy

@krumo For domain adaptation of detectors from the source domain to the target domain, we always train from the ImageNet pre-trained model. However, in practical application, the pre-trained model on the source domain is usually available. Why don't we fine-tune from the pre-trained model on the source domain model, but fine-tune from the ImageNet pre-trained model. The latter seems to take more time. Could you explain the reason for this?

Look forward to your reply.

Questions about experimental results

Hi, I have some questions regarding the performance of your code.
The original paper's performance ( using the SIM 10k dataset as the source domain and the Cityscapes training set as the target domain) is much lower than your code.

original:
图片

yours:
图片

I wonder why. Did you add some additional tricks or the implementation of original paper is buggy? Also, can your result be reproduced stably?
Wonderful work, by th way. Thanks a lot

Cannot reproduce the same results for Sim10k->Cityscapes task

Hi @krumo ,

Thanks for this great work, I have some questions.
I'm trying to reproduce the results of your example (adapting from Sim10k dataset to Cityscapes dataset)
I get higher value for second line (image-level alignment only) AP50 = 38.07 and a lower value for third line (ins-level alignment only) AP50=34.88
However AP50 values for the rest of the lines are acceptable.
I suppose there is a difference in hyperparameters for each experiment.

  1. could you please confirm if you use:
    DA_IMG_GRL_WEIGHT: 0.2
    DA_INS_GRL_WEIGHT: 0.2
    in all the experiments for this dataset. Or you use different DA_IMG_GRL_WEIGHT when you use the image-level alignment only and a different DA_INS_GRL_WEIGHT
    when you use the instance-level alignment only?

  2. to make sure that I do it correctly, when I want to do the second line experiment (image-level alignment only ), I just comment the lines:

    # Add Instance-level loss
    head_loss_gradients['instance'] = _add_instance_level_classifier(model, blob_feats_rois_all, dim_feats_rois_all)
    # Add consistency regularization
    head_loss_gradients['consistency'] = _add_consistency_loss(model, blob_conv, dim_conv, blob_feats_rois_all, dim_feats_rois_all)

    Is there anything else to do?

  3. you said in #1 you don't provide a configuration about weight loss, and the weights for two-level adaptation are the weights of GRL in two levels
    is that means DA_IMG_GRL_WEIGHT=DA_INS_GRL_WEIGHT= λ ?

λ is a trade-off parameter to balance the Faster R-CNN loss and our newly added domain adaptation components.

  1. if that true, where this DA_IMG_GRL_WEIGHT: 0.2 and DA_INS_GRL_WEIGHT: 0.2 came from? in the paper they didn't mention such a value.

we set λ= 0.1 for all experiments

  1. finally, in this case the L_cst is not multiplied by λ, but again in the paper

L=L_det+λ(L_img+L_ins+L_cst)

please correct me if i am wrong.

Thanks again

Reproducing results for Sim10k -> Cityscapes

Hi,
I downloaded the pretrained models: model_consist, model_imageinstances and ran test_net.py with the config 'e2e_da_faster_rcnn_vgg16-sim10k.yaml'. I am getting very low AP ( < 0.20) on Cityscapes car val, while getting about 0.33-0.34 on Sim10k.

I also trained the model using the same config 'e2e_da_faster_rcnn_vgg16-sim10k.yaml' with the converted VGG16 model downloaded from the link you provided. I'm getting AP 0.16 on Cityscapes car val and 0.40 for Sim10k.

Any idea on what might have gone wrong?

Also, I got several "Warning: invalid contours." while converting cityscapes to coco. Would it cause any issues?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.