Code Monkey home page Code Monkey logo

dualresidualnetworks's Introduction

Dual Residual Networks

By Xing Liu1, Masanori Suganuma1,2, Zhun Sun2, Takayuki Okatani1,2

Tohoku University1, RIKEN Center for AIP2

link to the paper

News

i) A summary of experimental settings for training is added.

ii) Some mistakes in ./train/raindrop.py are fixed.

Table of Contents

  1. Abstract

  2. Citation

  3. Numerical Results

  4. Models

  5. Datasets

  6. Test

  7. Train

  8. Visual Results

Abstract

In this paper, we study design of deep neural networks for tasks of image restoration. We propose a novel style of residual connections dubbed “dual residual connection”, which exploits the potential of paired operations, e.g., upand down-sampling or convolution with large- and smallsize kernels. We design a modular block implementing this connection style; it is equipped with two containers to which arbitrary paired operations are inserted. Adopting the “unraveled” view of the residual networks proposed by Veit et al., we point out that a stack of the proposed modular blocks allows the first operation in a block interact with the second operation in any subsequent blocks. Specifying the two operations in each of the stacked blocks, we build a complete network for each individual task of image restoration. We experimentally evaluate the proposed approach on five image restoration tasks using nine datasets. The results show that the proposed networks with properly chosen paired operations outperform previous methods on almost all of the tasks and datasets.

Citation

@inproceedings{DuRN_arxiv,
title={Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration},
author={Liu, Xing and Suganuma, Masanori and Sun, Zhun and Okatani, Takayuki},
booktitle={arXiv preprint arXiv:1903.08817},
year={2019},
}

@inproceedings{DuRN_cvpr19,
title={Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration},
author={Liu, Xing and Suganuma, Masanori and Sun, Zhun and Okatani, Takayuki},
booktitle={Proc. Conference on Computer Vision and Pattern Recognition},
pages={7007-7016},
year={2019},
}

Numerical results

Please find them in the test/results_confirmed.txt file.

Models

Please find them here.

Datasets

Gaussian noise removal

  • BSD500-gray (used in our paper)
    If you also want the original BSD500, click here.

Real-world noise removal

Motion blur removal

Haze removal

Raindrop removal

Rain-streak removal

Test

Requirements

  • python 3.7
  • pytorch 1.2.0

Instructions

  1. Download and un-zip the models, and put the trainedmodels in the project folder.
  2. Download and put the datasets into the data folder. Please also set the names for a dataset and its sub-folder(s) correctly, according to the current data folder.
  3. Go to the test folder, and run the scripts.

Visual results

Gaussian noise removal

Real-world noise removal

Motion blur removal - 1

Motion blur removal - 2

Some examples for object detection

Haze removal - 1

The images are taken by iphone 6 plus

Haze removal - 2

Haze removal - 3

Compare inside-feature maps with transmission map

Raindrop removal

Rain-streak removal

dualresidualnetworks's People

Contributors

dongheehand avatar evoliu avatar liu-vis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dualresidualnetworks's Issues

An error when run code

Hello, I try to run your code. But there is an error message.
error

And I want to know why.

the training code

Hi, thanks for your excellent jobs of restoration of multiple distortions .Have you finished the training code ? Thank you very much

Pretrain weights

Hi,

Thanks for you inspiring work. It would be great if you could share the pre-trained weights for each removals for us.

Thanks

about the additive Gaussian noise removal

Thanks for your interesting work! I want to ask about the task of additive Gaussian noise removal. Are the different levels of noise dataset ( i.e., 30, 50, 70) trained in a model together or trained separately for each level?

Test Raindrop error

I want to test the results on the raindrop removal task, I execute [python raindrop.py], but the result is as following.
And the PSNR and SSIM is not correct:
psnr: 10.630165165245023
ssim: 0.25498810946974426
1
0

some question about haze Removal

Thanks for good work.but I have a question,why you do not use norm layer in DuRN-US network in haze removal and use bacthnorm in rain steak removal?

Experimental environments

I am wondering about the experimental environments

  1. The number of gpus for training
  2. The gpu you used for training

Thank you!

version problem

Dear authors, hi.
There are some questions I want to ask you:
Your readme writes the environment is python3.7 + PyTorch 1.2, but when I clone your code, I find it has no different with the preceding version of code, which is python2.7 + PyTorch0.31;
Can you update your code version as you say?

关于感受野的问题

Hi你好,在阅读论文的过程中,我对这句话不是很理解:We also choose the kernel size and dilation rate for each DuRB so that the receptive field of convolution in each DuRB grows its size with L.
我尝试自己计算每一层的感受野,并没有发现有倍数关系。能麻烦您发我一份补充材料吗,下面是我的邮箱:[email protected]
不胜感激。

The PSNR metrics of different tasks

Hi, Dr Liu
Thanks for your great job!
I check the testing codes provided by this work, different tasks use different PSNR computation.
E.g.
(1) Denosing: the average channels of PSNR
(2) Deraining Drop: use the Y channel of images.
Could you please tell me that the reason about these implementations?
Thanks a lot!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.