britefury / cutmix-semisup-seg Goto Github PK

Great job for the semi-supervised semantic segmentation problem!

After checking the implementations, we do not find the related code to support running jobs with multiple GPUs,

We only find that the job function in the job_helper.py contains a parameter "num_jobs" as shown below,

Hi @Britefury,

Thank you for making the code publicly available.

Can you share the pretrained models for the deeplabv2 architecture (Cutmix, Cutout, VAT and ICT) please?
We are having troubles reproducing the results.

Thanks in advance.

Hi. In mean teacher, the consistency weight is 100. but in this work, all consistency weight is 1. Isn't this value too small?
Can you tell me the details of setting this parameter, because I have seen other work(such as
CPS, cvpr2021) that uses a consistency weight of around 100 when reproducing the mean-teacher method as well.

Looking forward to your help.

best,

Really nice work and super impressive results on the low-data regime (shown in Table 4, also pasted as following)!!

We have some small concerns about the red circle marked results:

1. On the 1/100 subset column, the baseline of DeepLabv3+/PSPNet is only 37.95%/36.69% while your method achieves 59.52%/67.20% separately. We really appreciate you if you could provide some detailed explanation of why your method could achieve so huge gains! One of our small concerns is that applying the CutMix scheme + Mean-Teacher scheme on the supervised baseline method w/o using unlabeled images might be a more reasonable baseline setting. It would be great if could share with us some results of such settings.

2. On the 1/8 subset column, the baseline performance of DeepLabv3+ is slightly worse than the PSPNet. According to our experience, the DeepLabv3+ should perform much better. Could you share with us some explanation on it?

3. On the full set column, we observe that the performance of the proposed method is slightly worse than the baseline. Could you share your comments on the possible reasons?

4. According to your code, all the experiments fix the BN statics and apply crop size 321x321 with a single GPU. Do you have any plans to train or have you ever trained your method on a more strong setting such as with crop size 512x512 + SyncBN + 8x V100 GPUs. MMSegmentation or openseg.pytorch might be a good candidate codebase.

Great thanks for your valuable time and wait for your explanation!

Hi. Thanks for your interesting work!

I have a question about the following code:

In the file of the main experiment, "mask_arr" is converted to "mask" and called by:

In the above code, a "loss_mask" is generated for unlabeled loss, i.e., the consistency loss. Are all elements of "loss_mask" equal to 1?
Can you explain what it does?

Thanks in advance.

Hi, really nice codebase!

We have tried to understand the details of your implementation and find the usage of the loss mask a little bit confusing (as shown below).

First, I have checked the code carefully and find that you define the 'mask' for the unlabeled images at the following code snippet:

Therefore, we can see that the initial 'mask' for all images is set as an array of 255 with the same size as the input image. Then you assign the mask value as following:

According to my understanding, only when we apply the cv2.warpAffine transform, some of the 'mask' values become zero (as shown below):

If my understanding is correct, the batch_um0 and batch_um1 might not be necessary if we do not apply the warpAffine transformation.

So my first question is about the influence of affine.cat_nx2x3 (shown as below) on the final performance.

So I guess the purpose of the loss mask is mainly to filter out the influence of the extra introduced pixels when applying the warpAffine augmentation. Is my understanding correct?

It would be great if you could point out any of my misunderstandings!

Thanks for your excellent work and kindly code releasing. I have a problem when reproducing the cityscapes experiment.

Following your tutorial, I have reproduced the experimental results, 44.37 vs. 51.73 with 6.79 improvement under the cutmix setting. When delving deep into the codes, I notice that you implement a downsample_label_img function for downsampling ground truth in the convert_cityscapes.py. However, it is more common to use nearest downsampling method in cv2 or PIL. Then, I replace the downsample_label_img function with the following:
y_img = cv2.resize(y_img, (1024, 512), interpolation=cv2.INTER_NEAREST)
and re-conduct the cityscapes experiment. The results degrades greatly, 43.79 vs. 47.02 with only 3.23 improvement.

I wonder the reason why a different downsample method affects the result greatly and the motivation to reimplement a downsample method rather than using a common one.

Hi, I went through the paper and had a tough time understanding the jist of it. Is it:

1. Cut-Mix for segmentation images improving the accuracy metrics or
2. The consistency loss function is improving your scores

In a more naive way, is the work about cut-mix for segmentation(orginal is for classification) or the work is about consistency loss

Hi, I found your paper really insightful, been trying to replicate the results. Thanks for the code and the elaborate explanations with training scripts. I was wondering if you could share the pretrained models for Cityscapes dataset (Cutout, Cutmix and baseline models) for comparison and evaluation.