Code Monkey home page Code Monkey logo

mi-aod's Introduction

MI-AOD

Language: 简体中文 | English

Python 3.7 PyTorch 1.6 CUDA 10.2 cuDNN 7.6.5 LICENSE

PWC PWC

Average time to resolve an issue Percentage of issues still open

Introduction

This is the code for Multiple Instance Active Learning for Object Detection, CVPR 2021.

Task Description

In this paper, we propose Multiple Instance Active Object Detection (MI-AOD), to select the most informative images for detector training by observing instance-level uncertainty.

The process of active object detection (active learning for object detection) is shown in the figure below.

Task

First, a small set of images X_L^0 (the labeled set) with instance labels Y_L^0 and a large set of images X_U^0 (the unlabeled set) without labels are given. For each image, the label consists of bounding boxes y_x^loc and categories y_x^cls for objects of interest.

A detection model M_0 is firstly initialized by using the labeled set {{X_L^0, Y_L^0}}. With the initialized model M_0, active learning targets at selecting a set of images X_S^0 from X_U^0 to be manually labeled and merging them with X_L^0 for a new labeled set X_L^1, i.e., X_L^1 = X_L^0 \union X_S^0. The selected image set X_S^0 should be the most informative, i.e., can improve the detection performance as much as possible.

The informativeness in the figure above is embodied as the uncertainty. That is to say, with the sample in X_U^0 input into the current model, if the output score of the model for each class is more uniform, the uncertainty of this sample is higher.

Based on the updated labeled set X_L^1, the task model is retrained and updated to M_1. The model training and sample selection repeat some cycles until the size of labeled set reaches the annotation budget.

Illustration

MI-AOD defines an instance uncertainty learning module, which leverages the discrepancy of two adversarial instance classifiers trained on the labeled set to predict instance uncertainty of the unlabeled set. MI-AOD treats unlabeled images as instance bags and feature anchors in images as instances, and estimates the image uncertainty by re-weighting instances in a multiple instance learning (MIL) fashion. Iterative instance uncertainty learning and re-weighting facilitate suppressing noisy instances, toward bridging the gap between instance uncertainty and image-level uncertainty.

Here and here are more paper interpretation in Chinese.

Illustration

Architecture

Innovation

  • Creatively design a tailor-made method for the active learning + object detection task for the first time.

  • Achieve 93.5% of the performance of 100% labeled data by using only 20% of the labeled data on the PASCAL VOC dataset.

  • Active learning was applied to the MS COCO dataset for the first time with the best performance.

  • The idea is clear and simple, and can be generalized to any types of detection models.

Results

Boarder Impact

MI-AOD focuses on object detection (OD), but it can be also generalized to:

  • any other visual object detection tasks (tiny object detection, pedestrian detection, medical image detection),
  • any other computer vision tasks (human pose detection, semantic/instance segmentation, temporal action detection),
  • and any other machine learning tasks like natural language processing,

by combining active learning with these tasks. This bottom-up and top-down idea can be generalized and applied to any of these tasks.

Notice that active learning works a lot for visual object detection in MI-AOD, other learning methods with less supervision can be combined with it, such as:

  • active few-shot learning,
  • active semi-/weak/self supervised learning,
  • active transfer learning,
  • active reinforcement learning,
  • active incremental learning,

and so on. These combination of active learning and other learning method can promote each other to a greater extent.

Getting Started

Installation

Please refer to Installation.md for installation.

Data Preparation

Please download VOC2007 datasets ( trainval + test ) and VOC2012 datasets ( trainval ) from:

VOC2007 ( trainval ): http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar

VOC2007 ( test ): http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar

VOC2012 ( trainval ): http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar

And after that, please ensure the file directory tree is as below:

├── VOCdevkit
│   ├── VOC2007
│   │   ├── Annotations
│   │   ├── ImageSets
│   │   ├── JPEGImages
│   ├── VOC2012
│   │   ├── Annotations
│   │   ├── ImageSets
│   │   ├── JPEGImages

You may also use the following commands directly:

cd $YOUR_DATASET_PATH
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
tar -xf VOCtrainval_06-Nov-2007.tar
tar -xf VOCtest_06-Nov-2007.tar
tar -xf VOCtrainval_11-May-2012.tar

If you want to use SSD detectors instead of RetinaNet detectors, you can replace the configuration file in this repository as below:

mv configs/MIAOD.py configs/MIAOD_Retina.py
mv configs/MIAOD_SSD.py configs/MIAOD.py

For the SSD detector, because the vgg16 pre-trained model link provided in the mmcv 1.0.5 package is no longer available, the json file that loads the pre-trained model link needs to be updated to the latest version:

wget https://github.com/open-mmlab/mmcv/raw/master/mmcv/model_zoo/open_mmlab.json
cp -v open_mmlab.json $YOUR_ANACONDA_PATH/envs/miaod/lib/python3.7/site-packages/mmcv/model_zoo/

Please change the $YOUR_ANACONDA_PATH to your actual Anaconda3 installation directory. Usually it would be ~/anaconda3.

After that, please modify the corresponding dataset directory, they are located in:

Line 2 of configs/MIAOD.py: data_root='$YOUR_DATASET_PATH/VOCdevkit/'
Line 2 of configs/_base_/voc0712.py: data_root='$YOUR_DATASET_PATH/VOCdevkit/'

Please change the $YOUR_DATASET_PATHs above to your actual dataset directory (i.e., the directory where you intend to put the downloaded VOC tar file).

And please use the absolute path (i.e., start with /) but not a relative path (i.e., start with ./ or ../).

Please refer to here for the information of other variables and parameters.

Please refer to here for the data preparation on MS COCO.

Train and Test

We recommend you to use a GPU but not a CPU to train and test, because it will greatly shorten the time.

And we also recommend you to use a single GPU, because the usage of multi-GPU may result in errors caused by the multi-processing of the dataloader.

However, thanks to @Kevin Chow, here is a feasible solution to train on multiple GPUs.

If you use only a single GPU, you can use the script.sh file directly as below:

chmod 700 ./script.sh
./script.sh $YOUR_GPU_ID

Please change the $YOUR_GPU_ID above to your actual GPU ID number (usually a non-negative number).

Please ignore the error if you run the script.sh file for the first time:

rm: cannot remove './log_nohup/nohup_$YOUR_GPU_ID.log': No such file or directory

The script.sh file will use the GPU with the ID number $YOUR_GPU_ID and PORT (30000+$YOUR_GPU_ID*100) to train and test.

The log file will not flush in the terminal, but will be saved and updated in the file log_nohup/nohup_$YOUR_GPU_ID.log and work_dirs/MI-AOD/$TIMESTAMP.log . These two logs are the same. You can change the directories and names of the latter log files in Line 48 of configs/MIAOD.py .

If you want to flush the log in the terminal, you can run these commands instead of using script.sh:

# for single GPU
python tools/train.py $CONFIG_PATH

# for multiple GPUs
tools/dist_train.sh $CONFIG_PATH $GPU_NUMBERS

where $CONFIG_PATH should be replaced by the path of the config file in the configs folder (usually it would be configs/MIAOD.py) and $GPU_NUMBERS should be replaced by the total numbers of used GPUs (it is not GPU ID number).

Similarly, theses commands are for test:

# for single GPU
python tools/test.py $CONFIG_PATH $CKPT_PATH --eval mAP

# for multiple GPUs
tools/dist_test.sh $CONFIG_PATH $CKPT_PATH $GPU_NUMBERS --eval mAP

where $CKPT_PATH should be replaced by the path of the checkpoint file (*.pth) in the work_dirs folder after training.

If you want to inference on single image, the commands are as follows:

python tools/test_single.py $CONFIG_PATH $CKPT_PATH $IMG_PATH $OUT_NAME

where $IMG_PATH should be replaced by the image on which you want to inference on, and $OUT_NAME should be replaced by the output result file name, which should usually end with .jpg, .png, and so on.

With output of an image with bounding boxes and scores, the uncertainty of this image will also be flushed in the terminal.

If you have any question, please feel free to leave a issue here.

And please refer to FAQ for frequently asked questions.

Code Structure

├── $YOUR_ANACONDA_DIRECTORY
│   ├── anaconda3
│   │   ├── envs
│   │   │   ├── miaod
│   │   │   │   ├── lib
│   │   │   │   │   ├── python3.7
│   │   │   │   │   │   ├── site-packages
│   │   │   │   │   │   │   ├── mmcv
│   │   │   │   │   │   │   │   ├── runner
│   │   │   │   │   │   │   │   │   ├── epoch_based_runner.py
│
├── ...
│
├── configs
│   ├── _base_
│   │   ├── default_runtime.py
│   │   ├── retinanet_r50_fpn.py
│   │   ├── voc0712.py
│   ├── MIAOD.py
│── log_nohup
├── mmdet
│   ├── apis
│   │   ├── __init__.py
│   │   ├── inference.py
│   │   ├── test.py
│   │   ├── train.py
│   ├── models
│   │   ├── dense_heads
│   │   │   ├── __init__.py
│   │   │   ├── MIAOD_head.py
│   │   │   ├── MIAOD_retina_head.py
│   │   │   ├── base_dense_head.py 
│   │   ├── detectors
│   │   │   ├── base.py
│   │   │   ├── single_stage.py
│   ├── utils
│   │   ├── active_datasets.py
├── tools
│   ├── test.py
│   ├── test_single.py
│   ├── train.py
├── work_dirs
│   ├── MI-AOD
├── script.sh

The code files and folders shown above are the main part of MI-AOD, while other code files and folders are created following MMDetection V2.3.0 to avoid potential problems.

The explanation of each code file or folder is as follows:

  • epoch_based_runner.py: Code for training and test in each epoch, which can be called by apis/train.py.

  • configs: Configuration folder, including running settings, model settings, dataset settings and other custom settings for active learning and MI-AOD.

    • __base__: Base configuration folder provided by MMDetection, which only need a little modification and then can be recalled by configs/MIAOD.py.

      • default_runtime.py: Configuration code for running settings, which can be called by configs/MIAOD.py.

      • retinanet_r50_fpn.py: Configuration code for model training and test settings, which can be called by configs/MIAOD.py.

      • voc0712.py: Configuration code for PASCAL VOC dataset settings and data preprocessing, which can be called by configs/MIAOD.py.

    • MIAOD.py: Configuration code in general including most custom settings, containing active learning dataset settings, model training and test parameter settings, custom hyper-parameter settings, log file and model saving settings, which can be mainly called by tools/train.py. The more detailed introduction of each parameter is in the comments of this file.

  • log_nohup: Log folder for storing log output on each GPU temporarily.

  • mmdet: The core code folder for MI-AOD, including intermidiate training code, object detectors and detection heads and active learning dataset establishment.

    • apis: The inner training, test and calculating uncertainty code folder of MI-AOD.

      • __init__.py: Some function initialization in the current folder.

      • inference.py: Code for inferring the model and calculating uncertainty, which can be called by tools/test_single.py.

      • test.py: Code for testing the model and calculating uncertainty, which can be called by epoch_based_runner.py, tools/test.py and tools/train.py.

      • train.py: Code for setting random seed and creating training dataloaders to prepare for the following epoch-level training, which can be called by tools/train.py.

    • models: The code folder with the details of network model architecture, training loss, forward propagation in test and calculating uncertainty.

      • dense_heads: The code folder of training loss and the network model architecture, especially the well-designed head architecture.

        • __init__.py: Some function initialization in the current folder.

        • MIAOD_head.py: Code for forwarding anchor-level model output, calculating anchor-level loss, generating pseudo labels and getting bounding boxes from existing model output in more details, which can be called by mmdet/models/dense_heads/base_dense_head.py and mmdet/models/detectors/single_stage.py.

        • MIAOD_retina_head.py: Code for building the MI-AOD model architecture, especially the well-designed head architecture, and define the forward output, which can be called by mmdet/models/dense_heads/MIAOD_head.py.

        • base_dense_head.py: Code for choosing different equations to calculate loss, which can be called by mmdet/models/detectors/single_stage.py.

      • detectors: The code folder of the forward propogation and backward propogation in the overall training, test and calculating uncertainty process.

        • base.py: Code for arranging training loss to print and returning the loss and image information, which can be called by epoch_based_runner.py.

        • single_stage.py: Code for extracting image features, getting bounding boxes from the model output and returning the loss, which can be called by mmdet/models/detectors/base.py.

    • utils: The code folder for creating active learning datasets.

      • active_dataset.py: Code for creating active learning datasets, including creating initial labeled set, creating the image name file for the labeled set and unlabeled set and updating the labeled set after each active learning cycle, which can be called by tools/train.py.
  • tools: The outer training and test code folder of MI-AOD.

    • test.py: Code for test on trained model and the whole test set for MI-AOD.

    • test_single.py: Code for test on trained model and single image in test set for MI-AOD.

    • train.py: Outer code for training and test for MI-AOD, including generating PASCAL VOC datasets for active learning, loading image sets and models, Instance Uncertainty Re-weighting and Informative Image Selection in general, which can be called by script.sh.

  • work_dirs: Work directory of the index and image name of the labeled set and unlabeled set for each cycle, all log and json outputs and the model state dictionary for the last 3 cycle, which are introduced in the Training and Test part above.

  • script.sh: The script to run MI-AOD on a single GPU. You can run it to train and test MI-AOD simply and directly mentioned in the Training and Test part above as long as you have prepared the conda environment and PASCAL VOC 2007+2012 datasets.

Model Zoo

Models

The trained model for the last cycle in active learning (i.e., using 20% labeled samples) are available on Google Drive and Baidu Drive (Extraction code: 1y9x).

Results

Results_RetinaNet_VOC

Proportion (%) of Labeled Images 5.0 7.5 10.0 12.5 15.0 17.5 20.0 100.0 (Full supervision)
mAP (%) of MI-AOD 47.18 58.41 64.02 67.72 69.79 71.07 72.27 77.28
Ratio (%) of the performance to full supervision 61.05 75.58 82.84 87.63 90.31 91.96 93.52 100.00

The training and test logs are available on Google Drive and Baidu Drive (Extraction code: 7a6m).

You can also use other files in the directory work_dirs/MI-AOD/ if you like, they are as follows:

  • JSON file $TIMESTAMP.log.json

    You can load the losses and mAPs during training and test from it more conveniently than from the work_dirs/MI-AOD/$TIMESTAMP.log file.

  • npy file X_L_$CYCLE.npy and X_U_$CYCLE.npy

    The $CYCLE is an integer from 0 to 6, which are the active learning cycles.

    You can load the indexes of the labeled set and unlabeled set for each cycle from them.

    The indexes are the integers from 0 to 16550 for PASCAL VOC datasets, where 0 to 5010 is for PASCAL VOC 2007 trainval set and 5011 to 16550 for PASCAL VOC 2012 trainval set.

    An example code for loading these files is the Line 108-114 in the tools/train.py file (which are in comments now).

  • pth file epoch_$EPOCH.pth and latest.pth

    The $EPOCH is an integer from 0 to 2, which are the epochs of the last label set training.

    You can load the model state dictionary from them.

    An example code for loading these files is the Line 109, 133-135 in the tools/train.py file (which are in comments now).

  • txt file trainval_L_07.txt, trainval_U_07.txt, trainval_L_12.txt and trainval_U_12.txt in each cycle$CYCLE directory

    The $CYCLE is the same as above.

    You can load the names of JPEG images of the labeled set and unlabeled set for each cycle from them.

    "L" is for the labeled set and "U" is for the unlabeled set. "07" is for the PASCAL VOC 2007 trainval set and "12" is for the PASCAL VOC 2012 trainval set.

An example output folder is provided on Google Drive and Baidu Drive (Extraction code: ztd6), including the log file, the last trained model, and all other files above.

Repository Contributor

In this repository, we reimplemented RetinaNet on PyTorch based on mmdetection. Thanks for their contribution.

License

This project is released under the Apache 2.0 license.

Citation

If you find this repository useful for your publications, please consider citing our paper.

@inproceedings{MIAOD2021,
    author    = {Tianning Yuan and
                 Fang Wan and
                 Mengying Fu and
                 Jianzhuang Liu and
                 Songcen Xu and
                 Xiangyang Ji and
                 Qixiang Ye},
    title     = {Multiple Instance Active Learning for Object Detection},
    booktitle = {CVPR},
    year      = {2021}
}

Stargazers repo roster for @yuantn/MI-AOD

mi-aod's People

Contributors

yuantn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

mi-aod's Issues

empty box in image

Hello!

I try to run experiments on my custom dataset, but the following error has occurred in here
IndexError: index 0 is out of bounds for dimension 0 with size 0

In my dataset there are images without bbox annotation, which cause the above error.
i.e., y_loc_img[0] = tensor([], device='cuda:0', size=(0, 4)) for images without bbox annotation.

What does the condition mean y_loc_img[0][0][0] < 0?

unlabeled使用gt信息

你好,MI-AOD的代码中的Unlabeled set也需要从dataset到dataloader,且在epoch_based_runner.py 70-74行涉及了对gt_bboxes和gt_labels的操作,与我理解的unlabeld set应该不使用gt信息有点出入,能否帮忙指点一下。

Train with Multi Gpu

当前代码在使用多卡训练时会出现 stop iteration的错,原因是某个卡上分配的数据比其他卡少,根本原因是由于在active_datasets.py中的create_X_L_file() 和 create_X_U_file(),多个卡会同时写同一个txt文件,导致先写完这个文件的卡创建dataloader时读取到了不全的txt.
解决方案:

  1. 在这两个函数中写文件时先随机sleep一小段时间,错开写文件的时间
    time.sleep(random.uniform(0,3))
    if not osp.exists(save_path):
        mmcv.mkdir_or_exist(save_folder)
        np.savetxt(save_path, ann[X_L_single], fmt='%s')
  1. 在tools/train.py中每次create_xx_file后,同步各个卡的线程。加上这句
          if dist.is_initialized():
              torch.distributed.barrier()

unlabeled 数据使用gt信息

你好,MI-AOD的代码中的Unlabeled set也需要从dataset到dataloader,且在epoch_based_runner.py 70-74行涉及了对gt_bboxes和gt_labels的操作,与我理解的unlabeld set应该不使用gt信息有点出入,能否帮忙指点一下。

Performance comparison of active object detection methods

Hello, I have a question about MI-AOD performance.

Looking at Fig. 5(a) in the paper, MI-AOD method is very good compared to other methods when trained with 5% of randomly selected images.
However, looking at (b) and (c), although MI-AOD is still a good method, there is no significant difference as (a) in the first cycle. Is there any special reason?

mi-aod question

Training error

I got the following error while running python tools/train.py configs/MIAOD.py

File "tools/train.py", line 257, in
main()
File "tools/train.py", line 131, in main
model = build_detector(cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)
File "/home/jobinkv/MI-AOD/mmdet/models/builder.py", line 67, in build_detector
return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/home/jobinkv/MI-AOD/mmdet/models/builder.py", line 32, in build
return build_from_cfg(cfg, registry, default_args)
File "/home/jobinkv/miniconda3/envs/miaod/lib/python3.7/site-packages/mmcv/utils/registry.py", line 167, in build_from_cfg
return obj_cls(**args)
File "/home/jobinkv/MI-AOD/mmdet/models/detectors/retinanet.py", line 17, in init
test_cfg, pretrained)
File "/home/jobinkv/MI-AOD/mmdet/models/detectors/single_stage.py", line 28, in init
self.init_weights(pretrained=pretrained)
File "/home/jobinkv/MI-AOD/mmdet/models/detectors/single_stage.py", line 38, in init_weights
self.backbone.init_weights(pretrained=pretrained)
File "/home/jobinkv/MI-AOD/mmdet/models/backbones/resnet.py", line 600, in init_weights
load_checkpoint(self, pretrained, strict=False, logger=logger)
File "/home/jobinkv/miniconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 224, in load_checkpoint
checkpoint = _load_checkpoint(filename, map_location)
File "/home/jobinkv/miniconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 175, in _load_checkpoint
model_urls = get_torchvision_models()
File "/home/jobinkv/miniconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 124, in get_torchvision_models
_zoo = import_module(f'torchvision.models.{name}')
File "/home/jobinkv/miniconda3/envs/miaod/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'torchvision.models.segmentation.data_loader'

训练自己的数据集问题

作者您好!
我在使用论文中的数据集时,可以正常的训练和测试,在换上自己的数据集时,可以进行第一轮的训练,但是在最小化的过程中,代码报了如下错误,望解答 QAQ
File "tools/train.py", line 267, in
main()
File "tools/train.py", line 203, in main
distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/apis/train.py", line 122, in train_detector
runner.run([data_loaders_L, data_loaders_U], cfg.workflow, cfg.total_epochs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 192, in run
epoch_runner([data_loaders[i], data_loaders_u[i]], **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 60, in train
outputs = self.model.train_step(X_L, self.optimizer, **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 31, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 228, in train_step
losses = self(**data)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/core/fp16/decorators.py", line 51, in new_func
return old_func(*args, **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 162, in forward
return self.forward_train(x, img_metas, **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/models/detectors/single_stage.py", line 83, in forward_train
losses = self.bbox_head.forward_train(x, img_metas, y_loc_img, y_cls_img, y_loc_img_ignore)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/models/dense_heads/base_dense_head.py", line 81, in forward_train
loss = self.L_wave_min(*loss_inputs, y_loc_img_ignore=y_loc_img_ignore)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/core/fp16/decorators.py", line 131, in new_func
return old_func(*args, **kwargs)
File "/home/zx/anaconda3/envs/active/lib/python3.7/site-packages/mmdet/models/dense_heads/MIAOD_head.py", line 479, in L_wave_min
if y_loc_img[0][0][0]< 0:
IndexError: index 0 is out of bounds for dimension 0 with size 0

Questions about script files

Hello, I want to run your code, but there was an error, so I am leaving a question.
When running ./script.sh $0 , the nohup_0.log file is created in the log_nohup folder, but it is empty.
After that, I run the script.sh file, it will not run, and also PYTHONPATH is not printed.
In addition, when running python tools/train.py and python mmdet/apis/train.py on terminal, there is no error and nothing is output.
What should I do in this case? I'd appreciate it if you could give me an answer.

low performance about SSD on PASCAL VOC compared to paper

image
如图,SSD on PASCAL VOC上的复现结果,与文章相比有明显下降,甚至不如random,想知道是否有其它同学也尝试复现,结果如何?

As shown in the figure, the reproduction results on SSD on Pascal VOC are significantly lower than those in the paper, and even worse than "random". I want to know whether other students also try to reproduce, and what are the results? Thank you!

missing 1 required positional argument: 'x'

encounter this error when I run python -m torch.distributed.launch --nproc_per_node=1 tools/train.py configs/MIAOD.py

2021-04-15 16:49:08,902 - mmdet - INFO - load model from: torchvision://resnet50
2021-04-15 16:49:09,276 - mmdet - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

fatal: Not a git repository (or any of the parent directories): .git
2021-04-15 16:49:14,869 - mmdet - INFO - Start running, host: z00495072@poweredge-server-01, work_dir: /home/z00495072/Projects/MI-AOD-master/work_dirs/MI-AOD/20210415_164906
2021-04-15 16:49:14,869 - mmdet - INFO - workflow: [('train', 1)], max: 3 epochs
Traceback (most recent call last):
  File "tools/train.py", line 267, in <module>
    main()
  File "tools/train.py", line 180, in main
    distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta)
  File "/home/z00495072/Projects/MI-AOD-master/mmdet/apis/train.py", line 120, in train_detector
    runner.run(data_loaders_L, cfg.workflow, cfg.total_epochs)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 122, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 32, in train
    **kwargs)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 31, in train_step
    return self.module.train_step(*inputs[0], **kwargs[0])
  File "/home/z00495072/Projects/MI-AOD-master/mmdet/models/detectors/base.py", line 228, in train_step
    losses = self(**data)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/z00495072/Projects/MI-AOD-master/mmdet/core/fp16/decorators.py", line 51, in new_func
    return old_func(*args, **kwargs)
TypeError: forward() missing 1 required positional argument: 'x'
Traceback (most recent call last):
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in <module>
    main()
  File "/home/z00495072/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/home/z00495072/anaconda3/envs/miaod/bin/python', '-u', 'tools/train.py', '--local_rank=0', 'configs/MIAOD.py']' returned non-zero exit status 1.

How to run using SSD model

请问是否有提供使用SSD网络的代码,以及对应的预训练模型是什么,文章中好像没有谈

关于模型保存的问题

作者您好!
在复现您的代码时,我发现之后最后3个epoch的model被保存,想请教您一下,如何能保存每个cycle的训练模型呢?

Comparison with other active learning methods

Hi,

you compared your method with 2 baselines (random, entropy) and 2 other approaches (Core-set, CDAL) and you outperformed all of them. The code with your query function is running but where in the code can i find the 4 other query functions to compare them with your approach for my own dataset?

Thanks in advance :)

Compared with semi-supervision

I would like to ask what is the difference between active learning and semi-supervision.

I am doing semi-supervised, I read the paper and the code and didn't find difference,
Is active learning and semi-supervised the same thing? Or What is the main difference between the two, and can I directly use active learning for semi-supervision?

MS COCO config

hi, i'm trying to reproduce your MS COCO results, could you update the MS COCO configs?

Many thanks!

Question about changing backbone

Hello, I have a question about changing the backbones.
I want to change the backbone retinanet to ssd.
( customize 3 just like you customized 1 to 2 according to the paper )

I tried customizing it, and the following error occurred. ( look at link4 )
TypeError: init() missing 1 required positional argument: 'input_size'

So when I deleted input_size, I got the following error.
NameError: name 'input_size' is not defined

model, train_cfg, test_cfg are well inputted.
How can I fix this?

  1. https://github.com/open-mmlab/mmdetection/blob/master/configs/_base_/models/retinanet_r50_fpn.py
  2. https://github.com/yuantn/MI-AOD/blob/master/configs/_base_/retinanet_r50_fpn.py
  3. https://github.com/open-mmlab/mmdetection/blob/master/configs/_base_/models/ssd300.py
  4. https://github.com/bluvory/kuaicv/blob/main/ssd_vgg_custom.py

Question about SSD on VOC

Hello,

Thank you for sharing works!

I ran MIAOD_SSD.py config to reproduce the Figure 5(b) in the paper.
But, the number of labeled images increases by 2k (not 1k) each cycle.

Questions about difference between code and paper

Question 1:
In def forward_single(self, x) of ~mmdet/dense_heads/MIAOD_retina_head.py​:
y_head_cls = y_head_f_mil.softmax(2) * y_head_cls_term2.sigmoid().max(2, keepdim=True)[0].softmax(1)
It seems that this code (with max(2, keepdim=True)[0]) is not in accordance with the equation (5) in the paper:
Screenshot from 2021-06-01 10-31-07
What is the function of max(2, keepdim=True)[0]? And why softmax(1) after all computation? I thought the code should be:
y_head_cls = (y_head_f_mil.softmax(2)) * (y_head_cls_term2.sigmoid().softmax(1))according to my understanding.

Question 2:
For uncertainty calculation definition in the paper:

The instance uncertainty is defined as the prediction discrepancy of f_1 and f_2.

and the equation for discrepancy is:

Screenshot from 2021-06-01 11-10-35

After reweighting, the equation has changed to:
Screenshot from 2021-06-01 11-10-01

I thought the code of uncertainty for loss computation and the code of uncertainty for data selection should be the same.
However, the code in def l_wave_dis of ~mmdet/dense_heads/MIAOD_head.py​:

l_det_cls_all = (abs(y_head_f_1_single - y_head_f_2_single) * w_i.reshape(-1, self.C)).mean(dim=1).sum() * self.param_lambda

directly compute the discrepancy by subtraction (Manhattan Distance?), while the code in def calculate_uncertainty of ~mmdet/apis/test.py​ is:

loss_l2_p = (y_head_f_1 - y_head_f_2).pow(2)

which uses pow(2) for square computation (Euclidean Distance?). I would like to know why you have such designs in code.

Thank you so much!

环境问题

Traceback (most recent call last):
File "./tools/train.py", line 15, in
from mmdet import version
File "/media/ubun/CC7251E47251D3B4/yoloseries/miaodxray/MI-AOD/mmdet/init.py", line 25, in
f'MMCV=={mmcv.version} is used but incompatible. '
AssertionError: MMCV==1.3.1 is used but incompatible. Please install mmcv>=1.0.5, <=1.0.5.
Traceback (most recent call last):
File "/home/ubun/anaconda3/envs/pytorch12/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/ubun/anaconda3/envs/pytorch12/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ubun/anaconda3/envs/pytorch12/lib/python3.6/site-packages/torch/distributed/launch.py", line 340, in
main()
File "/home/ubun/anaconda3/envs/pytorch12/lib/python3.6/site-packages/torch/distributed/launch.py", line 326, in main
sigkill_handler(signal.SIGTERM, None) # not coming back
File "/home/ubun/anaconda3/envs/pytorch12/lib/python3.6/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ubun/anaconda3/envs/pytorch12/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAOD.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
Killing subprocess 41043

Loss is nan

作者,您好!

当我使用自己数据集测试您代码时,l_det_loc、L_det损失是 nan,l_imgcls一直是0

自己数据集只有行人类,已经修改过datasets/voc.py中的CLASSES=('person',)

训练时使用的1个显卡

Stop Iteration when training on SSD and 9k labeled data

您好,当SSD on PASCAL VOC数据集打标签数据量为9000时,在rerweighting and minimizing部分的train_detector(model, [datasets, datasets_u], [cfg, cfg_u], distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta) 会报stop iteration的错误,请问是什么问题?是因为unlabelled set的大小小于labeled set了吗,怎么修改呢?

多卡训练,精度大幅度下降

作者您好,我使用多卡训练(4张),并且增大了你设置的默认单卡bathsize。

训练时,第一次测试mAP接近0.(单卡训练时,第一次一般是0.6)。

想请教一下,这是什么原因呀?是学习率的问题吗?

关于单张图像测试demo的问题,以及如何获取单张图像的不确定性

image
您好,在进行单张图像测试时,出现这样的错误,方便告知是什么原因嘛?另外,如果我是想获取单张图像的不确定性呢,是不是需要调整推理时result = model(return_loss=False, rescale=True, **data)为y_head_f_1, y_head_f_2, y_head_cls = model(return_loss=False, rescale=True, return_box=return_box, **data),之后再基于calculate_uncertainty函数做修改和调整!给你的工作和生活带来不便,还请谅解,期待您的回复!谢谢

训练中 第一个循环后报错 RuntimeError: CUDA error: no kernel image is available for execution on the device

google了相应的问题,发现mmdet中如果CUDA runtime version和 compiler version不匹配就会有这种报错。

然而问题是模型已经正常训练了一个循环,也生成了.npy的权重文件,但在进入下一个循环时突然报错。按理说不是会在正常训练前就报错吗?会不会是有什么其他的原因导致错误呢?

系统环境:
CentOS 6.3
CUDA9.0 Conda中安装了9.2的toolkit
其余按照安装说明操作。

log日志如下:
99.9%
99.9%
99.9%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
2021-08-21 21:24:18,266 - mmdet - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

Traceback (most recent call last):
File "./tools/train.py", line 257, in
main()
File "./tools/train.py", line 170, in main
distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta)
File "/home/disk2/vis/wrs/proj/MI-AOD/mmdet/apis/train.py", line 75, in train_detector
broadcast_buffers=False, find_unused_parameters=True)
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 333, in init
self.broadcast_bucket_size)
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 549, in _distributed_broadcast_coalesced
dist._broadcast_coalesced(self.process_group, tensors, buffer_size)
RuntimeError: CUDA error: no kernel image is available for execution on the device
Traceback (most recent call last):
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 261, in
main()
File "/home/disk2/vis/wrs/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 257, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/disk2/vis/wrs/envs/miaod/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAOD.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

关于mil分类器

作者您好,我拜读了您的关于面向目标检测的多实例主动学习方法论文,也复现了您基于mmdetection的代码,有个小问题向您请教,有关于mil分类器训练不是十分理解,一张图像可能有多个目标,比如同时有猫和狗,那么如何给这个图像整体的标签去训练这个整体分类器呢,我知道这个答案应该在公式6中,但是每个锚点的真实值可以给定,如何给定整张图像的标签呢

Validaiton| TypeError: 'DataContainer' object is not subscriptable

Encounter this error when procssessing validation

[ ] 0/4952, elapsed: 0s, ETA:Traceback (most recent call last):
File "./tools/train.py", line 267, in
main()
File "./tools/train.py", line 242, in main
distributed=distributed, validate=args.no_validate, timestamp=timestamp, meta=meta)
File "/home/bdggj/MI-AOD/mmdet/apis/train.py", line 120, in train_detector
runner.run(data_loaders_L, cfg.workflow, cfg.total_epochs)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 161, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 43, in train
self.call_hook('after_train_epoch')
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 298, in call_hook
getattr(hook, fn_name)(self)
File "/home/bdggj/MI-AOD/mmdet/core/evaluation/eval_hooks.py", line 71, in after_train_epoch
gpu_collect=self.gpu_collect)
File "/home/bdggj/MI-AOD/mmdet/apis/test.py", line 85, in multi_gpu_test
y_head = model(return_loss=False, rescale=True, **data)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/bdggj/MI-AOD/mmdet/core/fp16/decorators.py", line 51, in new_func
return old_func(*args, **kwargs)
File "/home/bdggj/MI-AOD/mmdet/models/detectors/base.py", line 164, in forward
return self.forward_test(x, img_metas, return_box=return_box, **kwargs)
File "/home/bdggj/MI-AOD/mmdet/models/detectors/base.py", line 144, in forward_test
return self.simple_test(imgs[0], img_metas[0], return_box=return_box, **kwargs)
File "/home/bdggj/MI-AOD/mmdet/models/detectors/single_stage.py", line 109, in simple_test
y_head_loc_cls = self.bbox_head.get_bboxes(*outs, img_metas, rescale=rescale)
File "/home/bdggj/MI-AOD/mmdet/core/fp16/decorators.py", line 131, in new_func
return old_func(*args, **kwargs)
File "/home/bdggj/MI-AOD/mmdet/models/dense_heads/MIAOD_head.py", line 641, in get_bboxes
img_shape = img_metas[img_id]['img_shape']
TypeError: 'DataContainer' object is not subscriptable
Traceback (most recent call last):
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 340, in
main()
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 326, in main
sigkill_handler(signal.SIGTERM, None) # not coming back
File "/home/bdggj/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/bdggj/anaconda3/envs/miaod/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAOD.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
Killing subprocess 30225

Unable to Download Pretrained SSD Model

When I decided to use the SSD model for this repo. I followed your instructions but the SSD Model is unreachable.
Here is my output:

2021-09-28 19:14:43,252 - mmdet - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.7.10 | packaged by conda-forge | (default, Sep 13 2021, 19:43:44) [GCC 9.4.0]
CUDA available: True
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 10.0, V10.0.130
GPU 0: Tesla T4
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.6.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.3-Product Build 20210617 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF, 

TorchVision: 0.7.0
OpenCV: 4.5.3
MMCV: 1.0.5
MMDetection: 2.3.0+cbed89d
MMDetection Compiler: GCC 7.3
MMDetection CUDA Compiler: 10.2
------------------------------------------------------------

2021-09-28 19:14:43,253 - mmdet - INFO - Distributed training: True
2021-09-28 19:14:44,676 - mmdet - INFO - Config:
input_size = 300
model = dict(
    type='SingleStageDetector',
    pretrained='open-mmlab://vgg16_caffe',
    backbone=dict(
        type='SSDVGG',
        input_size=300,
        depth=16,
        with_last_pool=False,
        ceil_mode=True,
        out_indices=(3, 4),
        out_feature_indices=(22, 34),
        l2_norm_scale=20),
    neck=None,
    bbox_head=dict(
        type='SSDHead',
        in_channels=(512, 1024, 512, 256, 256, 256),
        C=20,
        anchor_generator=dict(
            type='SSDAnchorGenerator',
            scale_major=False,
            input_size=300,
            basesize_ratio_range=(0.2, 0.9),
            strides=[8, 16, 32, 64, 100, 300],
            ratios=[[2], [2, 3], [2, 3], [2, 3], [2], [2]]),
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[0.1, 0.1, 0.2, 0.2])))
cudnn_benchmark = True
train_cfg = dict(
    assigner=dict(
        type='MaxIoUAssigner',
        pos_iou_thr=0.5,
        neg_iou_thr=0.5,
        min_pos_iou=0.0,
        ignore_iof_thr=-1,
        gt_max_assign_all=False),
    smoothl1_beta=1.0,
    allowed_border=-1,
    pos_weight=-1,
    neg_pos_ratio=3,
    debug=False,
    param_lambda=0.5)
test_cfg = dict(
    nms=dict(type='nms', iou_threshold=0.45),
    min_bbox_size=0,
    score_thr=0.02,
    max_per_img=200)
theta_f_1 = [
    'bbox_head.f_1_convs.0.weight', 'bbox_head.f_1_convs.0.bias',
    'bbox_head.f_1_convs.1.weight', 'bbox_head.f_1_convs.1.bias',
    'bbox_head.f_1_convs.2.weight', 'bbox_head.f_1_convs.2.bias',
    'bbox_head.f_1_convs.3.weight', 'bbox_head.f_1_convs.3.bias',
    'bbox_head.f_1_convs.4.weight', 'bbox_head.f_1_convs.4.bias',
    'bbox_head.f_1_convs.5.weight', 'bbox_head.f_1_convs.5.bias'
]
theta_f_2 = [
    'bbox_head.f_2_convs.0.weight', 'bbox_head.f_2_convs.0.bias',
    'bbox_head.f_2_convs.1.weight', 'bbox_head.f_2_convs.1.bias',
    'bbox_head.f_2_convs.2.weight', 'bbox_head.f_2_convs.2.bias',
    'bbox_head.f_2_convs.3.weight', 'bbox_head.f_2_convs.3.bias',
    'bbox_head.f_2_convs.4.weight', 'bbox_head.f_2_convs.4.bias',
    'bbox_head.f_2_convs.5.weight', 'bbox_head.f_2_convs.5.bias'
]
data_root = '/home/ubuntu/bahadir/datasets/VOCdevkit/'
dataset_type = 'VOCDataset'
img_norm_cfg = dict(mean=[123.675, 116.28, 103.53], std=[1, 1, 1], to_rgb=True)
train_pipeline = [
    dict(type='LoadImageFromFile', to_float32=True),
    dict(type='LoadAnnotations', with_bbox=True),
    dict(
        type='PhotoMetricDistortion',
        brightness_delta=32,
        contrast_range=(0.5, 1.5),
        saturation_range=(0.5, 1.5),
        hue_delta=18),
    dict(
        type='Expand',
        mean=[123.675, 116.28, 103.53],
        to_rgb=True,
        ratio_range=(1, 4)),
    dict(
        type='MinIoURandomCrop',
        min_ious=(0.1, 0.3, 0.5, 0.7, 0.9),
        min_crop_size=0.3),
    dict(type='Resize', img_scale=(300, 300), keep_ratio=False),
    dict(
        type='Normalize',
        mean=[123.675, 116.28, 103.53],
        std=[1, 1, 1],
        to_rgb=True),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(300, 300),
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=False),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[1, 1, 1],
                to_rgb=True),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img'])
        ])
]
data = dict(
    samples_per_gpu=8,
    workers_per_gpu=3,
    train=dict(
        type='RepeatDataset',
        times=1,
        dataset=dict(
            type='VOCDataset',
            ann_file=[
                '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/ImageSets/Main/trainval.txt',
                '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2012/ImageSets/Main/trainval.txt'
            ],
            img_prefix=[
                '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/',
                '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2012/'
            ],
            pipeline=[
                dict(type='LoadImageFromFile', to_float32=True),
                dict(type='LoadAnnotations', with_bbox=True),
                dict(
                    type='PhotoMetricDistortion',
                    brightness_delta=32,
                    contrast_range=(0.5, 1.5),
                    saturation_range=(0.5, 1.5),
                    hue_delta=18),
                dict(
                    type='Expand',
                    mean=[123.675, 116.28, 103.53],
                    to_rgb=True,
                    ratio_range=(1, 4)),
                dict(
                    type='MinIoURandomCrop',
                    min_ious=(0.1, 0.3, 0.5, 0.7, 0.9),
                    min_crop_size=0.3),
                dict(type='Resize', img_scale=(300, 300), keep_ratio=False),
                dict(
                    type='Normalize',
                    mean=[123.675, 116.28, 103.53],
                    std=[1, 1, 1],
                    to_rgb=True),
                dict(type='RandomFlip', flip_ratio=0.5),
                dict(type='DefaultFormatBundle'),
                dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
            ])),
    val=dict(
        type='VOCDataset',
        ann_file=
        '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt',
        img_prefix='/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(300, 300),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=False),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[1, 1, 1],
                        to_rgb=True),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]),
    test=dict(
        type='VOCDataset',
        ann_file=[
            '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/ImageSets/Main/trainval.txt',
            '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2012/ImageSets/Main/trainval.txt'
        ],
        img_prefix=[
            '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2007/',
            '/home/ubuntu/bahadir/datasets/VOCdevkit/VOC2012/'
        ],
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(300, 300),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=False),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[1, 1, 1],
                        to_rgb=True),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]))
evaluation = dict(interval=5, metric='mAP')
checkpoint_config = dict(interval=1)
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0005)
optimizer_config = dict()
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[1])
epoch_ratio = [5, 1]
epoch = 2
X_L_repeat = 16
X_U_repeat = 16
k = 10000
X_S_size = 1000
X_L_0_size = 1000
cycles = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
work_directory = './work_dirs/MI-AOD_SSD'
gpu_ids = range(0, 1)

2021-09-28 19:14:44,676 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-09-28 19:14:44,716 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-09-28 19:14:44,958 - mmdet - INFO - load model from: open-mmlab://vgg16_caffe
Downloading: "https://open-mmlab.s3.ap-northeast-2.amazonaws.com/pretrain/third_party/vgg16_caffe-292e1171.pth" to /home/ubuntu/.cache/torch/hub/checkpoints/vgg16_caffe-292e1171.pth
Traceback (most recent call last):
  File "./tools/train.py", line 257, in <module>
    main()
  File "./tools/train.py", line 131, in main
    model = build_detector(cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)
  File "/home/ubuntu/bahadir/MI-AOD-SSD/mmdet/models/builder.py", line 67, in build_detector
    return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
  File "/home/ubuntu/bahadir/MI-AOD-SSD/mmdet/models/builder.py", line 32, in build
    return build_from_cfg(cfg, registry, default_args)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/utils/registry.py", line 167, in build_from_cfg
    return obj_cls(**args)
  File "/home/ubuntu/bahadir/MI-AOD-SSD/mmdet/models/detectors/single_stage.py", line 28, in __init__
    self.init_weights(pretrained=pretrained)
  File "/home/ubuntu/bahadir/MI-AOD-SSD/mmdet/models/detectors/single_stage.py", line 38, in init_weights
    self.backbone.init_weights(pretrained=pretrained)
  File "/home/ubuntu/bahadir/MI-AOD-SSD/mmdet/models/backbones/ssd_vgg.py", line 84, in init_weights
    load_checkpoint(self, pretrained, strict=False, logger=logger)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 224, in load_checkpoint
    checkpoint = _load_checkpoint(filename, map_location)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 189, in _load_checkpoint
    checkpoint = load_url_dist(model_url)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 111, in load_url_dist
    checkpoint = model_zoo.load_url(url, model_dir=model_dir)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/hub.py", line 481, in load_state_dict_from_url
    download_url_to_file(url, cached_file, hash_prefix, progress=progress)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/hub.py", line 379, in download_url_to_file
    u = urlopen(req)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 261, in <module>
    main()
  File "/home/ubuntu/anaconda3/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 257, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ubuntu/anaconda3/envs/miaod/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAOD.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

AssertionError: Please specify at least one operation

AssertionError: Please specify at least one operation (save/eval/format/show the results / save the results) with the argument "--out", "--eval", "--format-only", "--show" or "--show-dir"

The problem came up when I was running this step**python tools/test.py $CONFIG_PATH $CKPT_PATH**.
I want to know what is $CONFIG_PATH and what is $CKPT_PATH.
How should I solve this problem?Thanks!

AttributeError: 'NoneType' object has no attribute 'param_lambda'

Hello, when I run dist_test.sh with the command: bash tools/dist_test.sh work_dirs/MIAL/20210316_092453/latest.pth 4 or run the file test.py using vscode。There are AttributeError: 'NoneType' object has no attribute 'param_lambda' error 。 I would like to ask you how I can solve this error。 My Torch version is 1.5 and TorchVision is 0.6.0 and MMCV version is 1.0.5

HI, maybe a small obstacle for the many, but what went wrong,can you help ? thanks

cycles = [0, 1, 2, 3, 4, 5, 6]
work_directory = './work_dirs/MIAL'
gpu_ids = range(0, 1)

2021-03-16 11:48:19,160 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-03-16 11:48:19,192 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-03-16 11:48:19,539 - mmdet - INFO - load model from: torchvision://resnet50
2021-03-16 11:48:19,698 - mmdet - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

2021-03-16 11:48:24,285 - mmdet - INFO - Start running, host: phzhou@ubuntu-MW51-HP0-00, work_dir: /disk1/huihui/MIAL/work_dirs/MIAL/20210316_114818
2021-03-16 11:48:24,285 - mmdet - INFO - workflow: [('train', 1)], max: 3 epochs
Traceback (most recent call last):
File "./tools/train.py", line 267, in
main()
File "./tools/train.py", line 180, in main
distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta)
File "/disk1/huihui/MIAL/mmdet/apis/train.py", line 120, in train_detector
runner.run(data_loaders_L, cfg.workflow, cfg.total_epochs)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 122, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 32, in train
**kwargs)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 36, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/disk1/huihui/MIAL/mmdet/models/detectors/base.py", line 228, in train_step
losses = self(**data)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/disk1/huihui/MIAL/mmdet/core/fp16/decorators.py", line 51, in new_func
return old_func(*args, **kwargs)
TypeError: forward() missing 1 required positional argument: 'x'
Traceback (most recent call last):
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/phzhou/anaconda3/envs/mial/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/phzhou/anaconda3/envs/mial/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAL.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

How to run it on COCO Dataset?

I have a problem, if i wante run it on coco dataset, if we need to fix the file<active_datasets.py>? because its way to get X_U and X_L only fix the standard VOC dataset.Hope for your reply,and if possible, could you provide the python file to fit the coco dataset? THX~

Loss is nan

作者,您好!

当我使用自己数据集测试您代码时,l_det_loc、L_det损失是 nan,l_imgcls一直是0

自己测试集只有行人类,已经修改过datasets/voc.py中的CLASSES=('person',)

训练时使用的1个显卡

关于分布式训练的问题

作者您好!
看到您的代码里通过script.sh和dist_train.sh,调用torch.distributed.launch实现分布式训练。
我想问如果单机单卡的话还需要分布式训练吗?是否直接Python train.py即可呢?

感谢您有意义的工作!

Train with the whole dataset

Hello!

I try to run experiments that gradually use the whole dataset.
But when the proportion of the labeled data have been used came to 1100/1659, I run into the StopIteration problem.
I was wondering how to set the config that could use the whole labeled dataset when all the active learning cycles finish.

Thanks in advance.

run error

hello, I want to run your code,but when i scanner" ./script.sh $0'',there is not any reaction,I fellow your instructions before ,I do not kown why?

(active_learn) root@interactive49465:/opt/data/private/ps/workspace/MI-AOD# ./script.sh $0
(active_learn) root@interactive49465:/opt/data/private/ps/workspace/MI-AOD#

When I am training my data, there is a StopIteration problem.

I want to know which parameters need to be modified when training my datasets.
I trained with my own ten picture data, but there was a problem, as shown in the figure.
2021-07-12 15-23-11屏幕截图
I don't know whether my parameters are not modified correctly or the training data is small.
Can you help me, please? Thank you.

AttributeError: 'Tensor' object has no attribute 'isnan'

Hello,
I want to try the code followed the README.md , but meet a AttributeError . This is my nohup_0.log.

2021-04-23 09:40:30,348 - mmdet - INFO - Environment info:

sys.platform: linux
Python: 3.7.10 (default, Feb 26 2021, 18:47:35) [GCC 7.3.0]
CUDA available: True
CUDA_HOME: /usr/local/cuda-10.0
NVCC: Cuda compilation tools, release 10.0, V10.0.130
GPU 0: GeForce GTX 1080 Ti
GCC: gcc (Ubuntu 5.3.1-14ubuntu2) 5.3.1 20160413
PyTorch: 1.4.0
PyTorch compiling details: PyTorch built with:

  • GCC 7.3
  • Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • NNPACK is enabled
  • CUDA Runtime 10.0
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  • CuDNN 7.6.3
  • Magma 2.5.1
  • Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

TorchVision: 0.5.0
OpenCV: 4.5.1
MMCV: 1.0.5
MMDetection: 2.3.0+b6976f3
MMDetection Compiler: GCC 5.3
MMDetection CUDA Compiler: 10.0

2021-04-23 09:40:30,349 - mmdet - INFO - Distributed training: True
2021-04-23 09:40:30,549 - mmdet - INFO - Config:
model = dict(
type='RetinaNet',
pretrained='torchvision://resnet50',
backbone=dict(
type='ResNet',
depth=50,
num_stages=4,
out_indices=(0, 1, 2, 3),
frozen_stages=1,
norm_cfg=dict(type='BN', requires_grad=True),
norm_eval=True,
style='pytorch'),
neck=dict(
type='FPN',
in_channels=[256, 512, 1024, 2048],
out_channels=256,
start_level=1,
add_extra_convs='on_input',
num_outs=5),
bbox_head=dict(
type='MIAODRetinaHead',
C=20,
in_channels=256,
stacked_convs=4,
feat_channels=256,
anchor_generator=dict(
type='AnchorGenerator',
octave_base_scale=4,
scales_per_octave=3,
ratios=[0.5, 1.0, 2.0],
strides=[8, 16, 32, 64, 128]),
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0.0, 0.0, 0.0, 0.0],
target_stds=[1.0, 1.0, 1.0, 1.0]),
FL=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
SmoothL1=dict(type='L1Loss', loss_weight=1.0)))
train_cfg = dict(
assigner=dict(
type='MaxIoUAssigner',
pos_iou_thr=0.5,
neg_iou_thr=0.4,
min_pos_iou=0,
ignore_iof_thr=-1),
allowed_border=-1,
pos_weight=-1,
debug=False,
param_lambda=0.5)
test_cfg = dict(
nms_pre=1000,
min_bbox_size=0,
score_thr=0.05,
nms=dict(type='nms', iou_threshold=0.5),
max_per_img=100)
data_root = '/data/database/VOCdevkit/'
dataset_type = 'VOCDataset'
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='Resize', img_scale=(1000, 600), keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1000, 600),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]
data = dict(
samples_per_gpu=2,
workers_per_gpu=2,
train=dict(
type='RepeatDataset',
times=3,
dataset=dict(
type='VOCDataset',
ann_file=[
'/data/database/VOCdevkit/VOC2007/ImageSets/Main/trainval.txt',
'/data/database/VOCdevkit/VOC2012/ImageSets/Main/trainval.txt'
],
img_prefix=[
'/data/database/VOCdevkit/VOC2007/',
'/data/database/VOCdevkit/VOC2012/'
],
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='Resize', img_scale=(1000, 600), keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
])),
val=dict(
type='VOCDataset',
ann_file='/data/database/VOCdevkit/VOC2007/ImageSets/Main/test.txt',
img_prefix='/data/database/VOCdevkit/VOC2007/',
pipeline=[
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1000, 600),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]),
test=dict(
type='VOCDataset',
ann_file=[
'/data/database/VOCdevkit/VOC2007/ImageSets/Main/trainval.txt',
'/data/database/VOCdevkit/VOC2012/ImageSets/Main/trainval.txt'
],
img_prefix=[
'/data/database/VOCdevkit/VOC2007/',
'/data/database/VOCdevkit/VOC2012/'
],
pipeline=[
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1000, 600),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]))
evaluation = dict(interval=3, metric='mAP')
checkpoint_config = dict(interval=1)
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(policy='step', step=[2])
epoch_ratio = [3, 1]
epoch = 2
X_L_repeat = 2
X_U_repeat = 2
k = 10000
X_S_size = 413
X_L_0_size = 827
cycles = [0, 1, 2, 3, 4, 5, 6]
work_directory = './work_dirs/MI-AOD'
gpu_ids = range(0, 1)

2021-04-23 09:40:30,549 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-04-23 09:40:30,695 - mmdet - INFO - Set random seed to 666, deterministic: False
2021-04-23 09:40:31,433 - mmdet - INFO - load model from: torchvision://resnet50
2021-04-23 09:40:31,718 - mmdet - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

2021-04-23 09:40:49,383 - mmdet - INFO - Start running, host: dreamtech@Dreamtech-Ubuntu, work_directory: /data/liudong/MI-AOD/work_dirs/MI-AOD/20210423_094030
2021-04-23 09:40:49,383 - mmdet - INFO - workflow: [('train', 1)], max: 3 epochs
2021-04-23 09:41:10,869 - mmdet - INFO - Epoch [1][50/827] lr: 1.000e-03, eta: 0:17:23, time: 0.429, data_time: 0.125, memory: 2472, l_det_cls: 1.1565, l_det_loc: 0.6727, l_imgcls: 0.2681, L_det: 2.0973
2021-04-23 09:41:24,112 - mmdet - INFO - Epoch [1][100/827] lr: 1.000e-03, eta: 0:13:46, time: 0.265, data_time: 0.003, memory: 2472, l_det_cls: 1.1592, l_det_loc: 0.6610, l_imgcls: 0.2422, L_det: 2.0624
2021-04-23 09:41:37,323 - mmdet - INFO - Epoch [1][150/827] lr: 1.000e-03, eta: 0:12:24, time: 0.264, data_time: 0.002, memory: 2472, l_det_cls: 1.1534, l_det_loc: 0.6440, l_imgcls: 0.2421, L_det: 2.0395
2021-04-23 09:41:50,599 - mmdet - INFO - Epoch [1][200/827] lr: 1.000e-03, eta: 0:11:38, time: 0.266, data_time: 0.002, memory: 2472, l_det_cls: 1.1565, l_det_loc: 0.6424, l_imgcls: 0.2262, L_det: 2.0251
2021-04-23 09:42:03,864 - mmdet - INFO - Epoch [1][250/827] lr: 1.000e-03, eta: 0:11:04, time: 0.265, data_time: 0.002, memory: 2472, l_det_cls: 1.1567, l_det_loc: 0.6499, l_imgcls: 0.2463, L_det: 2.0529
2021-04-23 09:42:17,229 - mmdet - INFO - Epoch [1][300/827] lr: 1.000e-03, eta: 0:10:38, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 1.1537, l_det_loc: 0.6276, l_imgcls: 0.2298, L_det: 2.0111
2021-04-23 09:42:30,514 - mmdet - INFO - Epoch [1][350/827] lr: 1.000e-03, eta: 0:10:15, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 1.1505, l_det_loc: 0.6406, l_imgcls: 0.2471, L_det: 2.0382
2021-04-23 09:42:43,814 - mmdet - INFO - Epoch [1][400/827] lr: 1.000e-03, eta: 0:09:55, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 1.1510, l_det_loc: 0.6212, l_imgcls: 0.2421, L_det: 2.0143
2021-04-23 09:42:57,238 - mmdet - INFO - Epoch [1][450/827] lr: 1.000e-03, eta: 0:09:37, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 1.1472, l_det_loc: 0.6159, l_imgcls: 0.2403, L_det: 2.0034
2021-04-23 09:43:10,702 - mmdet - INFO - Epoch [1][500/827] lr: 1.000e-03, eta: 0:09:19, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 1.1101, l_det_loc: 0.5917, l_imgcls: 0.2279, L_det: 1.9297
2021-04-23 09:43:24,124 - mmdet - INFO - Epoch [1][550/827] lr: 1.000e-03, eta: 0:09:03, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 1.0495, l_det_loc: 0.5960, l_imgcls: 0.2370, L_det: 1.8824
2021-04-23 09:43:37,522 - mmdet - INFO - Epoch [1][600/827] lr: 1.000e-03, eta: 0:08:47, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.9738, l_det_loc: 0.5769, l_imgcls: 0.2151, L_det: 1.7659
2021-04-23 09:43:50,945 - mmdet - INFO - Epoch [1][650/827] lr: 1.000e-03, eta: 0:08:31, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.9126, l_det_loc: 0.5775, l_imgcls: 0.2348, L_det: 1.7250
2021-04-23 09:44:04,240 - mmdet - INFO - Epoch [1][700/827] lr: 1.000e-03, eta: 0:08:15, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.8241, l_det_loc: 0.5727, l_imgcls: 0.2263, L_det: 1.6231
2021-04-23 09:44:17,694 - mmdet - INFO - Epoch [1][750/827] lr: 1.000e-03, eta: 0:08:00, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 0.9404, l_det_loc: 0.5767, l_imgcls: 0.2281, L_det: 1.7452
2021-04-23 09:44:31,113 - mmdet - INFO - Epoch [1][800/827] lr: 1.000e-03, eta: 0:07:45, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.7694, l_det_loc: 0.5403, l_imgcls: 0.2050, L_det: 1.5147
2021-04-23 09:44:39,094 - mmdet - INFO - Saving checkpoint at 1 epochs
2021-04-23 09:44:58,374 - mmdet - INFO - Epoch [2][50/827] lr: 1.000e-03, eta: 0:07:20, time: 0.381, data_time: 0.115, memory: 2472, l_det_cls: 0.7876, l_det_loc: 0.5425, l_imgcls: 0.2281, L_det: 1.5582
2021-04-23 09:45:11,835 - mmdet - INFO - Epoch [2][100/827] lr: 1.000e-03, eta: 0:07:06, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 0.7754, l_det_loc: 0.5320, l_imgcls: 0.2188, L_det: 1.5262
2021-04-23 09:45:25,274 - mmdet - INFO - Epoch [2][150/827] lr: 1.000e-03, eta: 0:06:52, time: 0.269, data_time: 0.002, memory: 2472, l_det_cls: 0.7756, l_det_loc: 0.5378, l_imgcls: 0.2221, L_det: 1.5355
2021-04-23 09:45:38,635 - mmdet - INFO - Epoch [2][200/827] lr: 1.000e-03, eta: 0:06:37, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.7371, l_det_loc: 0.5270, l_imgcls: 0.2007, L_det: 1.4648
2021-04-23 09:45:51,883 - mmdet - INFO - Epoch [2][250/827] lr: 1.000e-03, eta: 0:06:23, time: 0.265, data_time: 0.003, memory: 2472, l_det_cls: 0.8152, l_det_loc: 0.5017, l_imgcls: 0.2200, L_det: 1.5369
2021-04-23 09:46:05,192 - mmdet - INFO - Epoch [2][300/827] lr: 1.000e-03, eta: 0:06:09, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.8940, l_det_loc: 0.5325, l_imgcls: 0.2306, L_det: 1.6571
2021-04-23 09:46:18,661 - mmdet - INFO - Epoch [2][350/827] lr: 1.000e-03, eta: 0:05:55, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 0.7503, l_det_loc: 0.5077, l_imgcls: 0.2171, L_det: 1.4751
2021-04-23 09:46:31,928 - mmdet - INFO - Epoch [2][400/827] lr: 1.000e-03, eta: 0:05:41, time: 0.265, data_time: 0.003, memory: 2472, l_det_cls: 0.7683, l_det_loc: 0.5151, l_imgcls: 0.2229, L_det: 1.5063
2021-04-23 09:46:45,251 - mmdet - INFO - Epoch [2][450/827] lr: 1.000e-03, eta: 0:05:27, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.7067, l_det_loc: 0.4886, l_imgcls: 0.2108, L_det: 1.4061
2021-04-23 09:46:58,539 - mmdet - INFO - Epoch [2][500/827] lr: 1.000e-03, eta: 0:05:13, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.8194, l_det_loc: 0.4934, l_imgcls: 0.2091, L_det: 1.5219
2021-04-23 09:47:11,879 - mmdet - INFO - Epoch [2][550/827] lr: 1.000e-03, eta: 0:05:00, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.7372, l_det_loc: 0.4922, l_imgcls: 0.2097, L_det: 1.4391
2021-04-23 09:47:25,295 - mmdet - INFO - Epoch [2][600/827] lr: 1.000e-03, eta: 0:04:46, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.7582, l_det_loc: 0.4991, l_imgcls: 0.2144, L_det: 1.4717
2021-04-23 09:47:38,733 - mmdet - INFO - Epoch [2][650/827] lr: 1.000e-03, eta: 0:04:32, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 0.6970, l_det_loc: 0.4771, l_imgcls: 0.2037, L_det: 1.3777
2021-04-23 09:47:52,157 - mmdet - INFO - Epoch [2][700/827] lr: 1.000e-03, eta: 0:04:18, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.6835, l_det_loc: 0.4964, l_imgcls: 0.2033, L_det: 1.3831
2021-04-23 09:48:05,558 - mmdet - INFO - Epoch [2][750/827] lr: 1.000e-03, eta: 0:04:05, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.7212, l_det_loc: 0.4887, l_imgcls: 0.2004, L_det: 1.4103
2021-04-23 09:48:18,852 - mmdet - INFO - Epoch [2][800/827] lr: 1.000e-03, eta: 0:03:51, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.6734, l_det_loc: 0.4589, l_imgcls: 0.1959, L_det: 1.3282
2021-04-23 09:48:26,143 - mmdet - INFO - Saving checkpoint at 2 epochs
2021-04-23 09:48:45,242 - mmdet - INFO - Epoch [3][50/827] lr: 1.000e-03, eta: 0:03:29, time: 0.378, data_time: 0.114, memory: 2472, l_det_cls: 0.6987, l_det_loc: 0.4601, l_imgcls: 0.1850, L_det: 1.3438
2021-04-23 09:48:58,675 - mmdet - INFO - Epoch [3][100/827] lr: 1.000e-03, eta: 0:03:16, time: 0.269, data_time: 0.003, memory: 2472, l_det_cls: 0.6915, l_det_loc: 0.4607, l_imgcls: 0.1937, L_det: 1.3460
2021-04-23 09:49:11,901 - mmdet - INFO - Epoch [3][150/827] lr: 1.000e-03, eta: 0:03:02, time: 0.264, data_time: 0.003, memory: 2472, l_det_cls: 1.1146, l_det_loc: 0.5606, l_imgcls: 0.2404, L_det: 1.9156
2021-04-23 09:49:25,052 - mmdet - INFO - Epoch [3][200/827] lr: 1.000e-03, eta: 0:02:49, time: 0.263, data_time: 0.003, memory: 2472, l_det_cls: 1.1014, l_det_loc: 0.5852, l_imgcls: 0.2500, L_det: 1.9365
2021-04-23 09:49:38,401 - mmdet - INFO - Epoch [3][250/827] lr: 1.000e-03, eta: 0:02:35, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.9846, l_det_loc: 0.5921, l_imgcls: 0.2359, L_det: 1.8126
2021-04-23 09:49:51,723 - mmdet - INFO - Epoch [3][300/827] lr: 1.000e-03, eta: 0:02:22, time: 0.266, data_time: 0.003, memory: 2472, l_det_cls: 0.8661, l_det_loc: 0.5424, l_imgcls: 0.2485, L_det: 1.6571
2021-04-23 09:50:05,118 - mmdet - INFO - Epoch [3][350/827] lr: 1.000e-03, eta: 0:02:08, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.8491, l_det_loc: 0.5686, l_imgcls: 0.2555, L_det: 1.6733
2021-04-23 09:50:18,524 - mmdet - INFO - Epoch [3][400/827] lr: 1.000e-03, eta: 0:01:55, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.9489, l_det_loc: 0.5600, l_imgcls: 0.2339, L_det: 1.7427
2021-04-23 09:50:31,776 - mmdet - INFO - Epoch [3][450/827] lr: 1.000e-03, eta: 0:01:41, time: 0.265, data_time: 0.003, memory: 2472, l_det_cls: 0.8918, l_det_loc: 0.5770, l_imgcls: 0.2355, L_det: 1.7042
2021-04-23 09:50:45,165 - mmdet - INFO - Epoch [3][500/827] lr: 1.000e-03, eta: 0:01:28, time: 0.268, data_time: 0.003, memory: 2472, l_det_cls: 0.8749, l_det_loc: 0.5689, l_imgcls: 0.2469, L_det: 1.6907
2021-04-23 09:50:58,391 - mmdet - INFO - Epoch [3][550/827] lr: 1.000e-03, eta: 0:01:14, time: 0.265, data_time: 0.002, memory: 2472, l_det_cls: 0.8550, l_det_loc: 0.5952, l_imgcls: 0.2485, L_det: 1.6987
2021-04-23 09:51:11,735 - mmdet - INFO - Epoch [3][600/827] lr: 1.000e-03, eta: 0:01:01, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.8344, l_det_loc: 0.5818, l_imgcls: 0.2470, L_det: 1.6631
2021-04-23 09:51:25,076 - mmdet - INFO - Epoch [3][650/827] lr: 1.000e-03, eta: 0:00:47, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.8499, l_det_loc: 0.5668, l_imgcls: 0.2123, L_det: 1.6290
2021-04-23 09:51:38,406 - mmdet - INFO - Epoch [3][700/827] lr: 1.000e-03, eta: 0:00:34, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.8395, l_det_loc: 0.5657, l_imgcls: 0.2429, L_det: 1.6481
2021-04-23 09:51:51,771 - mmdet - INFO - Epoch [3][750/827] lr: 1.000e-03, eta: 0:00:20, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.8112, l_det_loc: 0.5347, l_imgcls: 0.2298, L_det: 1.5758
2021-04-23 09:52:05,099 - mmdet - INFO - Epoch [3][800/827] lr: 1.000e-03, eta: 0:00:07, time: 0.267, data_time: 0.003, memory: 2472, l_det_cls: 0.8015, l_det_loc: 0.5313, l_imgcls: 0.2344, L_det: 1.5672
2021-04-23 09:52:12,446 - mmdet - INFO - Saving checkpoint at 3 epochs
2021-04-23 09:52:16,575 - mmdet - INFO - Start running, host: dreamtech@Dreamtech-Ubuntu, work_directory: /data/liudong/MI-AOD/work_dirs/MI-AOD/20210423_094030
2021-04-23 09:52:16,575 - mmdet - INFO - workflow: [('train', 1)], max: 1 epochs
Traceback (most recent call last):
File "./tools/train.py", line 267, in
main()
File "./tools/train.py", line 203, in main
distributed=distributed, validate=(not args.no_validate), timestamp=timestamp, meta=meta)
File "/data/liudong/MI-AOD/mmdet/apis/train.py", line 122, in train_detector
runner.run([data_loaders_L, data_loaders_U], cfg.workflow, cfg.total_epochs)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 192, in run
epoch_runner([data_loaders[i], data_loaders_u[i]], **kwargs)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 78, in train
outputs = self.model.train_step(X_U, self.optimizer, **kwargs)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 36, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/data/liudong/MI-AOD/mmdet/models/detectors/base.py", line 228, in train_step
losses = self(**data)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/data/liudong/MI-AOD/mmdet/core/fp16/decorators.py", line 51, in new_func
return old_func(*args, **kwargs)
File "/data/liudong/MI-AOD/mmdet/models/detectors/base.py", line 162, in forward
return self.forward_train(x, img_metas, **kwargs)
File "/data/liudong/MI-AOD/mmdet/models/detectors/single_stage.py", line 83, in forward_train
losses = self.bbox_head.forward_train(x, img_metas, y_loc_img, y_cls_img, y_loc_img_ignore)
File "/data/liudong/MI-AOD/mmdet/models/dense_heads/base_dense_head.py", line 81, in forward_train
loss = self.L_wave_min(*loss_inputs, y_loc_img_ignore=y_loc_img_ignore)
File "/data/liudong/MI-AOD/mmdet/core/fp16/decorators.py", line 131, in new_func
return old_func(*args, **kwargs)
File "/data/liudong/MI-AOD/mmdet/models/dense_heads/MIAOD_head.py", line 483, in L_wave_min
if value.isnan():
AttributeError: 'Tensor' object has no attribute 'isnan'
Traceback (most recent call last):
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/dreamtech/.conda/envs/miaod/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/dreamtech/.conda/envs/miaod/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/MIAOD.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

Numbers used in the plot

Hi @yuantn

Thanks for your interesting work and the code!

I want to compare your work with others using SSD on PASCAL VOC.
But I cannot find the exact numbers used in Fig. 5(b) in your paper (CDAL and MI-AOD).
Could you share exact numbers (mean and standard deviation) used in Fig. 5(b), especially for MI-AOD and CDAL?
If you followed the LL4AL paper's experimental settings, you would have the mean and standard deviation for 3 trials.

I would appreciate if you could share the mean and standard deviation of CDAL and MI-AOD in Fig. 5(b).

Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.