chosj95 / mimo-unet Goto Github PK

Thanks for your great work.
When can you release the test code for the Realblur dataset?

Hello, thanks for your excellent work.
I have some confusions about the dataset you used for RealBlur.
Could you please tell me which dataset(s) did you use to obtain the results in Table 2, both GoPro and RealBlur or just only RealBlur itself.
MPRNET only used RealBlur to get the RealBlur results.

Thanks.

Hi,

Thanks for the great work. I didn't find the FLOPs of MIMO-Unet, MIMO-Unet+ in your paper. Could you please provide such numbers?

作者您好：
我想问下AFF融合模块的三个输入的维度以及是如何reshape操作的，期待您的回复

안녕하십니까,

화질 개선 연구를 진행하고 있는 석사과정 학생입니다. 선배님의 훌륭한 연구에 감사 드립니다.
올려주신 MIMO-UNet 모델을 Realblur-J데이터셋에서 PSNR을 확인한 결과 28.99가 나왔습니다.
제가 성능 평가 방식에 문제가 있는지 확인 해 주실 수 있으십니까?
dataset함수는 제가 직접 구현했으며, Reablur-J 데이터셋의 Test_list 파일을 통해 데이터를 불러오는 방식으로 구현했습니다.
PSNR은 올려주신 방식과 동일하게 구했으며, 이미지 사이즈가 8의 배수가 될 수 있도록 zero padding후 forward할 수 있도록했습니다. 이후 출력 이미지를 잘라서 Ground truth와 비교하여 평가했습니다.

감사합니다.

Tanks for your code. GOPRO_Large only have train and test dataset, how to valid

Hi @chosj95 , MIMO-UNet is a fantastic work! THX for sharing!
However, there are some problems confusing me a lot.

• During the training stage, the different size of blurry images are generated by nearest interpolation. However, the corresponding supervision of different size sharp images are generated by bilinear interpolation, rather than nearest interpolation again. Intuitively, the interpolation function should be the same.
• In the paper, function 7&8 have denominators in loss function for normalization. However, the code of loss function hasn't the denominators that I have mentioned above. I take it that chances are that the model will have better performance with normalization in loss function.

I am eagerly waitting for your explanation. THX a lot!

Hi,is this possibile to make the code for ssim to be public?

Hello, when I was training MIMO-UNet model, I needed a verification set after 100 training epochs. Could you tell me how to make the verification set? The error is as follows: No such file or directory: 'dataset/GOPRO\valid\blur\'

I test GoPro test dataset which has 1111 image pairs. It presents 32.45dB. Why it is lower than 32.68 dB proposed in paper?

In the Page 5 in paper (Figure 5. Several examples on the GoPro test dataset. )
I found the data in the Gopro dataset, the image is “test/blur/GOPR0854_11_00/001653_3.png” right?
I don't see the original image as blurry as the image shown in the paper.
Were you testing the image with the original Gopro dataset?

Hi,

Thanks for sharing this great work~ I found your implementation of loss here is different from the paper, where the "1/t_k" normalizationweight is not used in your code. Could you help clarify this?

result is not such good on a new dataset mixed by gopro and real blur

I encountered some problems when training the Go_Pro dataset on MIMO-UNet++. May I ask the author during the training process, will there be some unknown color blocks in the image generated at certain positions of the image?

Thank you for your great works.
I want to evaluate the images using your model with geometric self-ensemble.
Is there a testing code with self-ensemble?

Thanks

Hello, author, is your model suitable for mixed image restoration of motion blur and focus blur? Moreover, after training with motion blur data only, I tested the normal and focus blur images, and the results were extremely poor.

I have test the MIMO-UNet and MIMO-UNet+ on a single 2080Ti card (Theoretical performance is higher than TitanXp), which takes about 15ms and 30ms. I didn't make any changes to the open source code, just run the test command (https://github.com/chosj95/MIMO-UNet#test) directly.

python main.py --model_name "MIMO-UNet" --mode "test" --data_dir "dataset/GOPRO" --test_model "MIMO-UNet.pkl"

Namespace(batch_size=4, data_dir='dataset/GOPRO', gamma=0.5, learning_rate=0.0001, lr_steps=[500, 1000, 1500, 2000, 2500, 3000], mode='test', model_name='MIMO-UNet', model_save_dir='results/MIMO-UNet/weights/', num_epoch=3000, num_worker=8, print_freq=100, result_dir='results/MIMO-UNet/result_image/', resume='', save_freq=100, save_image=False, test_model='MIMO-UNet.pkl', valid_freq=100, weight_decay=0)

For MIMO-UNet:

==========================================================
The average PSNR is 31.73 dB
Average time: 0.015028

And for MIMO-UNet+

==========================================================
The average PSNR is 32.45 dB
Average time: 0.030238

1. Why the time of 8ms/17ms reported in the paper cannot be reproduced?
2. Why the asynchronous inference time on 2080Ti or 3090(https://github.com/chosj95/MIMO-UNet#gpu-syncronization-issue-on-measuring-inference-time) are slower than Titan XP(https://github.com/chosj95/MIMO-UNet#performance)?

In addition, I think the CUDA synchronized time should be used when reporting the time performance. The unsynchronized time can not correctly measure the speed and complexity of the model.

Could you please clarify your question? Are you asking if the GOPRO test set was resized to 256x256 during testing? I tried applying a center crop of 256x256 and found that it improved the results by nearly 0.5 points.

Hello, I ran the algorithm as you requested, but no prediction images appeared

In line 53 of eval.py(show as below), the output tensor is turned to PIL image after adding 0.5/255, then is saved. Why is this? This tensor has already been clipped to 0-1. I can't find the same processing way in the official document of torchvision

if args.save_image:
    save_name = os.path.join(args.result_dir, name[0])
    pred_clip += 0.5 / 255
    pred = F.to_pil_image(pred_clip.squeeze(0).cpu(), 'RGB')
    pred.save(save_name)

Hi~ @chosj95
Thanks for sharing your nice work and conducting careful comparison of the runtime performance. I want to compare my algorithm with the existing algorithms shown in the figure, could you please share the code and data for drawing this figure? Thanks a lot!

Hello,

How do you train the model for RealBlur?
Do you fine-tune on the GoPro pre-trained weights or train form scratch with GoPro+BSD+RealBlurR datasets like MPRNet?
And, do you plan to release the deblurring results of RealBlur_J and RealBlur_R?

Thank you for your great work!

I have a question on evaluation code.
In line 53 of eval.py:
pred_clip += 0.5 / 255
0.5/255 is added.
What does that mean? and the code calculates PSNR with unadded one(pred_numpy). Why is it?

Thank you.

what should be folder structure for testing the single image or group of images

when i use the nn.DataParallel(model), i got the problem as follows
Traceback (most recent call last):
File "main.py", line 67, in
main(args)
File "main.py", line 31, in main
_train(model, args)
File "MIMO-UNet/train.py", line 53, in _train
pred_img = model(input_img)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py", line 154, in forward
replicas = self.replicate(self.module, self.device_ids[:len(inputs)])
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py", line 159, in replicate
return replicate(module, device_ids, not torch.is_grad_enabled())
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/replicate.py", line 88, in replicate
param_copies = _broadcast_coalesced_reshape(params, devices, detach)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/replicate.py", line 71, in _broadcast_coalesced_reshape
tensor_copies = Broadcast.apply(devices, *tensors)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/_functions.py", line 21, in forward
outputs = comm.broadcast_coalesced(inputs, ctx.target_gpus)
File "/usr/local/lib/python3.6/dist-packages/torch/cuda/comm.py", line 39, in broadcast_coalesced
return torch._C._broadcast_coalesced(tensors, devices, buffer_size)
RuntimeError: inputs must be on unique devices

I get "RuntimeError: The size of tensor a (335) must match the size of tensor b (334) at non-singleton dimension 3" during testing on my own data. How to solve it?

Your work has been incredibly helpful to me, and I am very grateful. I was wondering if you could provide some information on the expected training time and required computing resources for the model. I would greatly appreciate it!

When you test SSIM ，did you use the function from skimage.metrics import structural_similarity？And set
structural_similarity(p_numpy, label_numpy, data_range=1，multichannel=True)

The valid_freq paramater is 100 in default. When your epoch is bigger than 100, you need to create a folder in GOPRO/tarin below, and put some sharp imgs in it.

Hello. Thanks for your great work and code.
I train it using the Tesla V100 16G, and it takes about 4 hours for 50 epochs.
Could you please tell me how long it took you to train for 3000 epochs?
Thanks in advance.

i want to deblur some images, but the resulting image is nearly the same as the input images.

Thanks for your greatest work！
When can you publish the code of the SSIM indicator?

Hello! Thanks for your great work!
Could you provide a checkpoint for a model trained on RealBlurJ+GOPRO dataset?

Thanks.

Does anyone reproduce the results given in the paper. I train mimo-unet, but it only reaches 31.40 on PSNR, which is 0.33 away from the 31.73 given in the paper.

Hello!
I'd like to know how to train MIMO_UNet on the realblur dataset.
Are any other modifications needed besides the parameters mentioned in the paper?
For example, is the patch size still 256*256?

Looking forward to your response. Thank you!

Thank you for sharing your great code. 😺

What is the license for this model? I'd like to cite it to the repository I'm working on if possible, but I want to post the license correctly.
https://github.com/PINTO0309/PINTO_model_zoo

Thank you.