hi，hologerry， thanks for open source the code, I just wonder does use a detector trained on object detection dataset to generate proposal can get a better result than selective search, as detector is better than selective search, I just wonder how important the proposal generation method's performance to SoCo framework, detector vs selective search.

Hi, hologerry.
Thanks for your wonderful work. I'm a rookie and try running all the code. I use a part of ImageNet due to insufficient computing resources. I successfully run the part 'prepare data with selective search' and got imagenet_filtered_proposals/train_ratio3size0308post.json and imagenet_root_proposals_mp/train (.pkl) finally. But then when I do the 'Pretrain with SoCo', I'm in some trouble.
I have some questions for you:

In 'SoCo_FPN_100ep.sh', data_dir="./data/ImageNet-Zip", What is the data format in this file?

And, When will the data(.json and .pkl) be used?

Thank you.

hi hologerry,

after using your 3-step selective search tool, I found the obtained bbox data is about 2.3T for imagenet, which is hard to store. Is this data size normal or I need some other important operation to filter the bbox?

Hi, thanks for your great work!
I notice that in your code the FastRCNNConvFCHead is composed of 4 Conv layers and 1 FC layer, but in FasterRCNN there are only 2 FC layers in the roi box head. Why is it designed this way?

hi，hologerry， thanks for your work, when we run the 'bash SoCo_C4_100ep.sh'
There is an error as follows. Our environment is pytorch 1.10,torchvision 0.11,GPU:A100,CUDA 11.4.
Thanks for your reply.

Traceback (most recent call last):
File "../main_pretrain.py", line 280, in
main(opt)
File "../main_pretrain.py", line 185, in main
train(epoch, train_loader, model, optimizer, scheduler, args, summary_writer)
File "../main_pretrain.py", line 204, in train
for idx, data in enumerate(train_loader):
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
<zipfile.ZipFile filename='../../imagenet/traintry.zip' mode='r'>
data.reraise()
File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
path raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/workspace/SoCo-main/contrast/data/dataset.py", line 506, in getitem
img2_1x_cut = self.transform[5](img2_1x, resized_bboxs2) # cutout
TypeError: call() missing 1 required positional argument: 'view_size'

Thank you for sharing your amazing work.

Do you have the mmdetection configs for finetuning the detectors?

Hi,

If I understand correctly, the scale aware assignment is handled by the correspondence matrices (corres_12, corres_13, etc.).
From the ablations in your paper I can see that you train a model without it but I'm not sure how that translates into code.
Do I adjust the correspondence matrices to disable this, and if so, how? It's unclear from the paper if this means assigning the proposals to all pyramid levels or just one, or something else entirely.

Thanks,
Linus

hi, hologerry~

I'm currently run your code on 4 V100 32G. I found it took about 1.3s for each iteration (batch size =128/GPU), thus the total training time for 100 epochs is about 7 days.

Does 1.3s sounds normal for you? I ran MoCo on same machines, and it took about 0.5s/iteration.

I'd appreciate if you can help me with this~
Thanks!

Hi,

I've managed to run the SoCo_FPN_100ep model and subsequently evaluate it on COCO using the provided configs. The performance I achieve is 39.8 bb AP and 36.0 mk AP.

I've checked that my training hyperparameters are the same as yours (as reported in the google drive config.json/log.txt). The only difference is that I ran mine on 8xV100 instead of 16. This should therefore give similar results to your Table 5.b where batch size is 1024, so 41.9 bb AP and 37.6 mk AP.

Do you have any idea why my numbers are lower? Any help would be appreciated.

Thanks,
Linus

Hi @hologerry ,

Thanks for your contribution, but I'm very confused about the data format requirements of this code, there doesn't seem to be any explanation.

I construct the data format like this:

| imagenet
| ---- train
| --------n01440766
| ---- val

However, when I try to run the code

bash ./tools/SoCo_FPN_100ep.sh

There is an error and I have no idea how to fix it:

FileNotFoundError: [Errno 2] No such file or directory: './data/imagenet/train_map.txt'

Could you please refine the README and provide the available data formats?

Hi，thanks for the great work. Is there any plan to release the pretrained models of Reset101? Thanks in advance.

It seems that jitter_prob is not initialized by args.jitter_prob in contrast/data/init.py, and it will always be the default value 0. Is it because the BoxJitter operation has a trivial improvement according to Table4 in the paper and we can ignore it?

Hi ,hologerry.
Thank you very much for your work!
I am very interested in understanding the organization of the dataset and would like to get your response.

In getitem func of class ImageFolderImageAsymBboxAwareMultiJitter1Cutout:

the cutout is used : img2_cutout = self.transform[7](img2, resized_bboxs2).

however, if flip operation is applied, the resized_bboxs2 is not aligned with bboxs2 and img2, is it a bug?

Does it should be img2_cutout = self.transform[7](img2, flip_box(resized_bboxs2)) ? (flip_box is just a pseudo func)

Thanks for your wonderful work. I think you are aware of finetuning Detectron with a longer time, 4x in the paper. Do you plan to train longer time? Say, 6x as in the paper "Rethinking ImageNet Pre-training" or longer until the model converges, and compare with random initialization to show the generalization and regularization characteristics of your method?
Thank you.

I'm very interested in your work.
Do you have a detailed code description?
For example, how to fine tune to coco dataset？

Hi, thanks for your great work and your code!

I notice that Sync BN was used in your models (e.g. SoCo_FPN_400ep), and there are SynBN operations in Backbone, FPN and RoI Head. So when I try to use your pretrained models like SoCo_FPN_400ep to do COCO-detection Task (say, faster_rcnn_fpn_res50), there could be three different choices about BN when finetuning on COCO-detection:

Possibly,

1. load all BN parameters (Backbone, FPN and RoI head) from SoCo_FPN_400ep, fix all of them and use normal BN when finetuning on COCO
2. load all BN parameters (Backbone, FPN and RoI head) from SoCo_FPN_400ep, fix all of them and also use Sync BN when finetuning on COCO
3. load all BN parameters (Backbone, FPN and RoI head) from SoCo_FPN_400ep, set them as training parameters (do not fix) and use normal BN when finetuning on COCO

Could you please give me some suggestions? many thanks in advance!

Best,

Thanks for sharing such a good work!
I am confused about the train configuration, base learning rate in this repository is 0.03 but 1.0 in paper, and weight decay in this repository is 0.000025 but 0.00001 in paper. Do these two different configurations result in significant performance changes?

Can you provide the following requirement.txt file ？

First of all, thank you for sharing the code. Can you tell me the organization under the imagenet_root file Jia?

Thanks for your great job~
In SoCo/detectron2_configs/R_50_C4_1x.yaml, Base-RCNN-C4-BN.yaml was refered.

Hello, thanks for your exciting work.
May I ask you when the code of HifaFace will be released?

Thank you for your great job!
I noticed that you added the result with the model which was pretrained on MS-COCO dataset recently in the revised version.
Could you please upload such a model or release the training script if it is convenient for you?
Thank you so much~ Looking forward to your reply.

Hello! Thanks for your work. Could you please provide more information on the mini-coco experiment? I want to reproduce it, but I couldn't find train splits and training hyperparameters.

Hi, Thanks for your great work!

I am recently learning your paper, and notice that some configs are different between SoCo paper and SoCo code.

Say, in SoCo paper, Table 1, it says that lr_base=1.0, wd=1e-5 and LARS are used. In SoCo code, lr_base=0.03, wd=1e-4 and SGD are used. If I want to use LARS optimizer, need I decrease the weight_decay ? Thanks in advance!

In Table 1, BYOL pretrained with 300 epoch achieves 40.4 APbbox. May I also ask that, was BYOL reproduced by your own in Table 1 and could you please give me some hints on settings (e.g. batch size, lr, wd) of reproducing BYOL results on COCO? (I'm now doing a project and really struggling on reproducing BYOL)

Looking forward to your answers and your many excellent works in the future!

Hi，hologerry， thanks for open source the code。
when I run bash toos/SoCo_C4_100ep.sh,the bug is:
contrast/data/dataset.py", line 371, in getitem
img = self.transform(image)
TypeError: 'tuple' object is not callable

Hi，thanks for the great work. Is there any plan to release the pretrained models of Reset101? Thanks in advance.

During my model training, the training log shows that the loss values are negative, and the learning rate remains at 0. But the loss is still decreasing. Is this normal?