liming-ai / aligndet Goto Github PK

Hello, here I want to test the effect of Faster-R CNN fine tuning, but do not see the weight parameter? Is it this final model.pth file? What is the meaning of selfsup_mask-rcnn_mstrain-soft-teacher_sampler-4096_temp0.5

Hello, I would like to ask whether fc_class and fc_reg are retained in the final pre-training model (whether the final finetune results in the paper retain the above two parts)?

Very nice and useful work! I want to pre-train a model for a custom dataset, so how can I get filtered proposals for my own dataset? Can you release the code for it? Thank you very much.

Hi, thanks for your impressive work!
I'm trying to run DETR on the VOC dataset, but I notice that only coco's post-clustering proposal is provided([email protected]_cluster-id-as-class.json). So how can I generate my own post clustering proposal? Is there any code available or more detailed steps for this please, thanks!

Have you tried experimenting with AlignDet in the yolo series?

How does DETR perform alignment operations?
Can I find a detailed description in the paper?
Where is the code?

nice work, thank you!

When I run:tools/dist_train.sh configs/selfsup/mask_rcnn.py 8 --work-dir work_dirs/selfsup_mask-rcnn
Q1:i Think Image-domain and Box-domain pre-training only update Neck&Head,didn't update backbone,is ture?
Q2:AlignDet only pre-training a model's Neck&Head.Backbone is frozen whole training process.Is true?
Q3:Backbone is only update in fine-tune process，is ture?
Thank you

`raceback (most recent call last):
File "D:\miniconda\envs\my_aligndet\lib\site-packages\mmcv\utils\misc.py", line 73, in import_modules_from_strings
imported_tmp = import_module(imp)
File "D:\miniconda\envs\my_aligndet\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'AlignDet'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\lyw\lyw_weed\AlignDet\tools\train.py", line 242, in
main()
cfg = Config.fromfile(args.config)
File "D:\miniconda\envs\my_aligndet\lib\site-packages\mmcv\utils\config.py", line 343, in fromfile
import_modules_from_strings(**cfg_dict['custom_imports'])
File "D:\miniconda\envs\my_aligndet\lib\site-packages\mmcv\utils\misc.py", line 80, in import_modules_from_strings
raise ImportError
ImportError`

After finetune, I want to perform testing and inference on mmdetection,When I run python demo/image_demo.py demo/demo.jpg./configs/mask_rcnn/mask-rcnn_r50_fpn_1x_coco.py --weights in mmdetection ./best_bbox_mAP_epoch_12.pth --device cpu, the message is :missing keys in source state_dict. How should I convert the AlignDet trained model to the mmdetection corresponding model

Hello, can your algorithm be trained with mixed precision, Does it support mixed precision training? I tried to modify the parameters, but it doesn't seem to work

Hello, there was an error when I was training with a single gpu，RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

A great work.
I have a little problem understanding the experiments in Table2. My understanding is as follows:

1. For those unaligned pre-training methods, only use 1%, 5%, ..., 100% data chosen from COCO train2017 to finetune them. Is that correct?
2. For those aligned pre-training methods, first pre-train(box-domain) them on the whole COCO train2017, then use 1%, 5%, ..., 100% data from COCO train2017 to finetune them. Is that correct?
3. In addition to the above two questions, I notice that in section4.2, there is a statement: "We provide 5 different
   data folds for each low-data setting, and the final performance is the average of all results". How to understand the number '5' and 'low-data' here? A single result in Table2 is the average of 5 results??? I am confused about the logics here.

Appreciation for answering my questions!

After I finish pre-training and fine-tuning, how do I get the performance metrics of the model?

When I use the configuration "configs selfsup doormaple_detr. py" for training, some errors occur:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/registry.py", line 69, in build_from_cfg
return obj_cls(**args)
File "/nas/xxxx/mmdet/aligndet/code/AlignDet/models/detectors/selfsup_detector.py", line 33, in init
self.online_backbone = build_detector(backbone)
File "/opt/conda/lib/python3.7/site-packages/mmdet/models/builder.py", line 59, in build_detector
cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/opt/conda/lib/python3.7/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/registry.py", line 72, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
AttributeError: SelfSupDeformableDETR: SelfSupDeformableDETRHead: 'ContrastiveLoss' object has no attribute 'use_sigmoid'

please help me, thank you!

Hello, this research is truly valuable and interesting. What I am curious about is whether the authors have tried using the ground truth annotations from the COCO training set as proposals for pre-training. Since the COCO annotations are more accurate, I guess using more accurate annotations might improve the effectiveness of pre-training.

How do I get filtered_proposals on my own dataset？

I have heard you: the Hungarian Algorithm needing explicit class information to achieve one-one matching, and the performance is poor if there is no pseudo-semantic labels. so I think the cls_cost is important, but you set it 0, why?

ModuleNotFoundError: No module named 'AlignDet'.

I read the questions on the discussion board，use SOCO code to get my custom dataset's filtered_proposals. SOCO code as this:

Prepare data with Selective Search

1. Generate Selective Search proposals

   python selective_search/generate_imagenet_ss_proposals.py

2. Filter out invalid proposals with filter strategy

   python selective_search/filter_ss_proposals_json.py

3. Post preprocessing for images of no proposals

   python selective_search/filter_ss_proposals_json_post_no_prop.py

However, I found that my [email protected] format are not the same as the coco dataset you provided(Google Drive). How can i get the right format of [email protected] to use Aligndet training . Could you provide this code? Where did I go wrong？ Thank you very much!

do you run the experiments when training DETR without using cluster results as the pseudo-semantic labels? is the perfermance degrade much?
thank you!

As shown in the figure above, in the results of this table, does the pre-training Data except backbone include the 1% Data that is fine-tuned

final-model.pth 能否用于YOLO框架下做与训练模型呢？

Hi, thanks for the great work! 😊

I am interested in how well weights pre-trained with AlignDet transfer to downstream tasks, which I consider to be an important reason for self-supervised detection pre-training. A standard procedure when transfer learning to downstream tasks is to pre-train a detector on MS COCO, OpenImagesv7, or Objects365 and use these weights to initialize all modules except for the classification head.

Have you tried out how AlignDet matches up against supervised pre-training on small to medium sized datasets? In your transfer learning experiment on Pascal VOC, do you compare AlignDet against a detector with supervised pre-training for all modules (except the classification head) or does the baseline train all modules but the backbone from scratch?

Hello，dear author，it's a nice work！I have been thoroughly studying your article and delving into the associated code and implementations. And i have a small question
In Section 3.2, specifically in the Box-domain Contrastive Learning part, I have a question regarding the following statement:

"For each query box q ∈ Q, assuming its assigned proposal index is i and the feature is zq. The set of positive keys K+ and negative keys K− for query feature zq can be constructed as: Z_{+} = \left{z \in Z_{1}:l=i \right}"

Upon examining your code, I noticed that there might be a correction needed in this statement. It seems that the correct formulation could be:

"Z_{+} = \left{z \in Z_{2}:l=i \right}"

Here is the relevant section of your original code: \models\roi_heads\bbox_heads\convfc_bbox_head.py line 142

# ... (previous code)
for label in torch.unique(online_labels):
    # ignore the background class
    if label == self.num_classes:
        continue

    query_inds = (online_labels == label) * (online_label_weights > 0)
    key_inds   = (target_labels == label) * (target_label_weights > 0)
    online_neg_inds = (online_labels != label) * (online_label_weights > 0)
    target_neg_inds = (target_labels != label) * (target_label_weights > 0)

    num_valid_labels += 1

    query = cls_online[query_inds]
    key = cls_target[key_inds] if key_inds.sum() > 0 else query
    neg = torch.cat([cls_online[online_neg_inds], cls_target[target_neg_inds]])

    loss_cls_ = loss_cls_ + self.loss_cls(query, key, neg, avg_factor=avg_factor)

losses['loss_cls'] = loss_cls_ / num_valid_labels
# ... (remaining code)

I wanted to bring this to your attention and seek clarification. If I have misunderstood something, please guide me in the right direction. I greatly appreciate your work and expertise in this field.

Best regards