Transfer Learning from Synthetic to Real-Noise Denoising with Adaptive Instance Normalization (Accepted for CVPR 2020)
Yoonsik Kim, Jae Woong Soh, Gu Yong Park, and Nam Ik Cho
[Arxiv]
- Ubuntu 16.04
- Tensorflow 1.8
- CUDA 9.0 & cuDNN 7.1
- Python 3.6
Real-noise denoising is a challenging task because the statistics of real-noise do not follow the normal distribution, and they are also spatially and temporally changing. In order to cope with various and complex real-noise, we propose a well-generalized denoising architecture and a transfer learning scheme. Specifically, we adopt an adaptive instance normalization to build a denoiser, which can regularize the feature map and prevent the network from overfitting to the training set. We also introduce a transfer learning scheme that transfers knowledge learned from synthetic-noise data to the real-noise denoiser. From the proposed transfer learning, the synthetic-noise denoiser can learn general features from various synthetic-noise data, and the real-noise denoiser can learn the real-noise characteristics from real data. From the experiments, we find that the proposed denoising method has great generalization ability, such that our network trained with synthetic-noise achieves the best performance for Darmstadt Noise Dataset (DND) among the methods from published papers. We can also see that the proposed transfer learning scheme robustly works for real-noise images through the learning with a very small number of labeled data.
We propose a novel well-generalized denoiser based on the AIN, which enables the CNN to work for various noise from many camera devices.
We introduce a transfer learning for the denoising scheme, which learns the domain-invariant information from synthetic noise (SN) data and updates affine transform parameters of AIN for the different-domain data.Average PSNR of the denoised images on the DND (left) and SIDD (right) benchmarks, we denote the environment of training, i.e., training with SN data only, RN data only, and both.
Will be updated soon.
Our work and implementations is inspired by and based on SPADE [site].