Thanks for sharing your work!
I wonder if there any update? And could you give the dependency of this project. Thanks!

I find that the forward need 4 parameter ,' query_rgb','support_rgb','support_mask','history_mask',
what's mean of the 'support_rgb','support_mask'? how can i get it , if it is any image with label?

when i use the network to predict a image , how modify the code ?

Thanks a lot!

Thanks for your contribution, and i want to compare with CANet in my paper.

Hello

The file for Binary_amp_aug.zip does not extract properly is there an issue with it?

Thanks

I read this article, and this method is very meaningful. Can you send me a code?

Hello, thanks you for your code. Is there a chance to upload more code such as network for 5-shot? I have been looking forward to your update.

Hello,

Thank you for your work.

Could you provide the trained weights of your model?

Ahyun Seo

In the train stage, the IOU is constant like this:

e:198 loss = 0.1104-:0.0000: 100%|██████████████| 15/15 [00:04<00:00, 3.53it/s]
----Evaluation----
IOU:0.0000
IOU for this epoch: 0.0000
best epoch:0 ,iout:0.0000
This epoch taks: 4.778814792633057 second
still need hour:0.0027
e:199 loss = 0.1190-:0.0000: 100%|██████████████| 15/15 [00:04<00:00, 3.51it/s]
----Evaluation----
IOU:0.0000
IOU for this epoch: 0.0000
best epoch:0 ,iout:0.0000
This epoch taks: 4.783169269561768 second
still need hour:0.0013

Anyone can thnow why?

My image is much bigger than the training image，so the pred image has a lot of noise，when I resize my images，the results are not accurate enough。
I found that your code only used the IOM once，could you give me some advice on my problem？

python version: 3.7.9
pip version: 20.3.3

I run the following command

python .\train.py

And get the following error

File ".\train.py", line 167, in <module>
    for i_iter, batch in enumerate(tqdm_gen):
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\tqdm\std.py", line 1166, in __iter__
    for obj in iterable:
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 345, in __next__
    data = self._next_data()
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 856, in _next_data
    return self._process_data(data)
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\dataloader.py", line 881, in _process_data
    data.reraise()
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\_utils.py", line 395, in reraise
    raise self.exc_type(msg)
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\_utils\worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\avh\Desktop\proj\dnn\CaNet\dataset_mask_train.py", line 95, in __getitem__
    os.path.join(self.data_dir, 'JPEGImages', support_name + '.jpg'))))))
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\PIL\Image.py", line 2904, in open
    fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '.\\JPEGImages\\2010_004817.jpg'

As good as I understand, there should be folder JPEGImages, with a lot of images. Can you provide example of that folder?

The work is very interesting, I am very much looking forward to the release of the code. Can I have a copy of the code for reference?

Hello
Thanks a lot for sharing the code. I am a bit confused about the setup you used for the metatraining stage. So during the validation you are using the validation data to determine best checkpoint (which in the few-shot setup case would be used as the hold out test set to sample both support and query images). Can you please correct me if there is something I misunderstood in your code and whether this is the correct setup you are using for the paper?

Also how are you evaluating the final numbers reported in your paper to compare with the SOA? Are you using the same exact OSLSM samples pairs ?

Greatly appreciate your efforts.

Thanks for sharing your code. The multi-scale testing result of fold 0 is 52.5 mIOU according to your paper but I can only get 51~ mIOU. Could you give me some advice on how to reproduce the same result as yours? Below is my multi-scale testing code modified from train.py.

with torch.no_grad():
    print ('----Evaluation----')
    model = model.eval()
    valset.history_mask_list=[None] * 1000
    all_inter, all_union, all_predict = [0] * 5, [0] * 5, [0] * 5
    for i_iter, batch in enumerate(valloader):

        query_rgb, query_mask, support_rgb, support_mask, history_mask, sample_class, index = batch
        query_rgb = (query_rgb).cuda(0)
        support_rgb = (support_rgb).cuda(0)
        support_mask = (support_mask).cuda(0)
        query_mask = (query_mask).cuda(0).long()  # change formation for crossentropy use

        query_mask = query_mask[:, 0, :, :]  # remove the second dim,change formation for crossentropy use
        history_mask = (history_mask).cuda(0)
        pred_softmax = torch.zeros(1,2,query_rgb.size(-2),query_rgb.size(-1)).cuda(0)
        for scale in [0.7,1,1.3]:
            query_= nn.functional.interpolate(query_rgb,scale_factor=scale,mode='bilinear',align_corners=True)
            scale_pred = model(query_, support_rgb, support_mask,history_mask)
            scale_pred_softmax = F.softmax(scale_pred, dim=1)
            pred_softmax += nn.functional.interpolate(scale_pred_softmax,size=query_rgb.size()[-2:], mode='bilinear',
                                         align_corners=True)
        pred_softmax/=3.
        # update history mask
        for j in range(support_mask.shape[0]):
            sub_index = index[j]
            valset.history_mask_list[sub_index] = pred_softmax[j]

#         pred = nn.functional.interpolate(pred, size=query_rgb.size()[-2:], mode='bilinear',
#                                          align_corners=True)  #upsample  # upsample

        _, pred_label = torch.max(pred_softmax, 1)
        inter_list, union_list, _, num_predict_list = get_iou_v1(query_mask, pred_label)
        for j in range(query_mask.shape[0]):#batch size
            all_inter[sample_class[j] - (options.fold * 5 + 1)] += inter_list[j]
            all_union[sample_class[j] - (options.fold * 5 + 1)] += union_list[j]

    IOU = [0] * 5

    for j in range(5):
        IOU[j] = all_inter[j] / all_union[j]

    mean_iou = np.mean(IOU)
    print('IOU:%.4f' % (mean_iou))

Thanks very much.

Hi! @icoz69 Thanks for your open sourced code. It is a very nice work ! Could you provide the detailed setting on COCO subset ?

Can you update the code for 5-shot semantic segmentation?Thank you very much!!!

Thanks for sharing your works. i use a quick evaluation as your code on fold 2. the meanIoU is only 48.7% (51.3% in article) and multi-scale cannot improve the preformance either. Is there any detail i have missed?
code is the same with that in train.py

The cvpr conference has already finished, may I know when the code will be released please?

There is no network code, would you mind to update the code and add the network.py

Thanks

Thank you for releasing the code
But I want to know how to test the trained model? I cannot find the test data or related code.

How to understand the parameter as follow in your code??

parser.add_argument('-fold',
type=int,
help='fold',
default=1)

hi when i am trying to train its showing

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

can you help me with this

In the paper you note that: "Our final result is achieved by bilinearly upsampling the confidence map to the same spatial size of the query image and classifying each location according to the confidence maps." As far as I can tell from the code, your model is not upsampling (the predictions come from here:

How are you evaluating the results when the predictions and labels are of different shapes? Do you downsize the masks to be the size of the predictions or upsample the predictions to be the size of the masks in other unreleased code?

Also, during training and eval, do you first resize the images and masks before passing them into the graph? Or do you use original pascal image dimensions?

This could relate to difficulties in producing reported results #4

Error when the code runs to ‘turn off’ function, should model.module be modified to the model？ The pytorch model does not seem to have the '.module' attribute, only the '.modules'.

Hi, I have been trying to reimplement this paper and cannot get the reported accuracy. It would be super helpful if you can give me some advice about the following details please:

1. Most importantly, would you mind to explain how the episodes are sampled from the dataset, and the definition of epoch mentioned in section 5.1 in more details please? (it would be very helpful if you can provide the total number of samples in the dataset and any data augmentation or preprocessing methods applied)
   Currently, I understand one epoch as using all masks available in the dataset as the support target once. However:
   If I include all class masks available in PASCAL VOC 2012 and SDS extra annotation into the dataset, the model will overfit after iterating through the dataset for 200 epochs (end up with meanIOU around 35%).
   If I do class balancing (take n random masks from each class, where n equals to the number of masks belong to the class with the least number of masks), the meanIOU on validation set will fluctuate a lot (by about 4%) between each epoch even when approach the end of training.
2. The training schedule
3. Are there any batchnorm layers in the model? If so, may I know where are they please?
4. Are the Dense Comparison Module output features passed through the residual block in the very first iteration when there hasn't been any predicted masks? If so, may I know what is the empty mask like please?
5. When doing testing, there will be an about 2% fluctuation in meanIOU introduced by the random sampling of 1000 support-query pair. May I know how do you deal with this fluctuation please?

Thanks in advance

Hi, @icoz69 . It seems that you exclude some data sample of PASCAL-5i in your experiment.
In the directory 'Binary_map_aug/train/1', it contains 586 pictures (***.png).
But in the 'Binary_map_aug/train/1.txt', it only contains 528 file names.
Can you please correct me if there is something I misunderstood in your data setting?

Hi, thank you for share your awesome work. But I cannot find the attention module and code for 5-shot semantic segmentation, can you tell me where is is?