This is a repository for Team SNU_CVLab, the winner of NTIRE2017 Challenge on Single Image Super-Resolution.
Our paper "Enhanced Deep Residual Networks for Single Image Super-Resolution" (PDF) won the Best Paper Award of the NTIRE workshop.

Team members: Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee from Computer Vision Lab, SNU
The codes are based on Facebook ResNet.

Structure of our single-scale model (EDSR).

Structure of our multi-scale model (MDSR).

In MDSR, every convolution layer execpt pre-processing modules uses 3x3 convolution kernel with stride = 1, padding = 1. Each pre-processing module has two residual blocks with 5x5 convolution kernels.

Our team (SNU_CVLab) ranked at 1st (EDSR) and 2nd (MDSR) place.

• torch7
• cudnn
• nccl (Optional, for faster GPU communication)

Clone this repository into $makeReposit:

makeReposit = [/the/directory/as/you/wish]
mkdir -p $makeReposit/; cd $makeReposit/
git clone https://github.com/LimBee/NTIRE2017.git

Please download the dataset from below. If you download the dataset in the other location, you have to change the optional argument -dataset when running NTIRE2017/code/main.lua file.

• DIV2K provided by NTIRE2017

  makeData = /var/tmp/dataset/ # Please set the absolute path as desired
  mkdir -p $makeData/; cd $makedata/
  # Please download the dataset from [CodaLab](https://competitions.codalab.org/competitions/16303)
  # Alternatively, you can download it from our server: http://cv.snu.ac.kr/research/EDSR/DIV2K.tar
  tar -xvf DIV2K.tar

  After untar, you will have the following directory structure:

  /var/tmp/dataset/DIV2K/DIV2K_train_HR/0???.png
  /var/tmp/dataset/DIV2K/DIV2K_train_LR_bicubic/X?/0???.png
  /var/tmp/dataset/DIV2K/DIV2K_train_LR_unknown/X?/0???.png
  

• Flickr2K collected by SNU_CVLab using Flickr API

  makeData = /var/tmp/dataset/
  mkdir -p $makeData/; cd $makedata/
  wget http://cv.snu.ac.kr/research/EDSR/Flickr2K.tar
  tar -xvf Flickr2K.tar

  After untar, you will have the following directory structure:

  /var/tmp/dataset/Flickr2K/Flickr2K_HR/00????.png
  /var/tmp/dataset/Flickr2K/Flickr2K_train_LR_bicubic/X?/00????x?.png
  /var/tmp/dataset/Flickr2K/Flickr2K_train_LR_unknown/X?/00????x?.png
  

  To generate input of unknown track, we trained simple downsampler network.
  Downsamplers can be downloaded by

  wget htt://cv.snu.ac.kr/research/EDSR/downsamplers.tar

To enable faster data loading, please convert the downloaded dataset into .t7 files

• To train with DIV2K

  cd $makeReposit/NTIRE2017/code/tools
  
  #This command generates multiple t7 files for
  #each images in DIV2K_train_HR folder
  th png_to_t7.lua -apath $makeData -dataset DIV2K -split true
  
  # This command generates a single t7 file that contains
  # every image in DIV2K_train_HR folder (Requires ~16GB RAM for training)
  th png_to_t7.lua -apath $makeData -dataset DIV2K -split false

• To train with Flickr2K

  cd $makeReposit/NTIRE2017/code/tools
  
  # This command generates multiple t7 files for
  # each images in Flickr2K_HR folder
  th png_to_t7.lua -apath $makeData -dataset Flickr2K -split true

You can choose to use .png filesm too. Details are described below.

You can download our pre-trained models and super-resolve your own image.

1. Download our models into $makeReposit/NTIRE2017/demo/model/

   cd $makdReposit/NTIRE2017/demo/model/
   # Our models submitted to the Challenge
   wget http://cv.snu.ac.kr/research/EDSR/model_challenge.tar
   # Our models used to write the paper
   wget http://cv.snu.ac.kr/research/EDSR/model_paper.tar

   We used 2 model ensemble for unknown track of challenge.

2. Run test.lua with given models and images:

   cd $makeReposit/NTIRE2017/demo
   th test.lua -selfEnsemble true      # This command runs our final bicubic_x2 model

   You can reproduce our final results with provided shell script makeFinal.sh in NTIRE2017/demo directory. You have to uncomment the appropriate line before you run.

   sh makeFinal.sh

   • Here are some optional arguments you can adjust. If you have any problem in running above examples, please refer following line.

     -nGPU     [n]   # You can test our model with multiple GPU. (n = 1, 2, 4)
     
     -dataDir  [$makeData]           #Please specify this directory. Default is /var/tmp/dataset
     -type     [bench | test | val]
     -dataset  [DIV2K | myData]
     -save     [Folder name]
     
     -selfEnsemble [true | false]        # Generates 8 output images using single model.
                                         # Do not use this option for unknown downsampling.
     
     -chopSize [S]   # Please reduce the chopSize when test fails due to GPU memory.
                     # The optimal size of S can be vary depend on your maximum GPU memory.
     
     -progress [true | false]

     -type val uses down-sampled DIV2K 0791~0800 to generate the output. Please note that we did not used those images for training. You can check how different options affect the total performance by runnning evaluation after you generated sets of validation images. (Requires MATLAB)

     matlab -nodisplay <evaluation.m

     If you do not want to calculate SSIM, please modify evaluation.m file as below. Calculating PSNR only will take a few seconds.

     line 6:     psnrOnly = false; -> psnrOnly = true;
     

   You can test with your own model and images. Just put your images in NTIRE2017/demo/img_input directory. If you have ground-truth high-resolution images for your own dataset, please locate them in NTIRE2017/demo/img_target/myData to evaluate the results.

   th test.lua -type test -dataset myData -model anyModel -scale [2 | 3 | 4] -degrade [bicubic | unknown]

   Soon, we will support some famous super-resolution benchmark sets like Set5, Set14, Unban100, BSD100.

1. To train our shallow baseline model, please run the following command:

   th main.lua         # This model is not our final model!

   • Here are some optional arguments you can adjust. If you have any problem in running above examples, please refer following line. For more information about arguments, please refer to NTIRE2017/code/opts.lua file.

     -nGPU     [n]   # You can train expert model with multiple GPU. (Not multiscale model.)
     -nThreads [n]   # Number of threads for data loading.
     
     -datadir [$makeData]  # Please specify this directory. Default is /var/tmp/dataset
     
     -save [Folder name]     # You can generate experiment folder with given name.
     -load [Folder name]     # You can resume your experiment from the last checkpoint.
                             # Please do not set -save and -load at the same time.
     
     -datatype   [png | t7 | t7pack]     # png < t7 < t7pack - requires larger memory
                                         # png > t7 > t7pack - requires faster CPU & Storage
     
     -chopSize   [S]     # Please reduce the chopSize when test fails due to GPU memory.
                         # The optimal size of S can be vary depend on your maximum GPU memory.

2. To train our final model, please use the provided shell script training.sh in NTIRE2017/code directory. You have to uncomment the appropriate line before you run.

   Some model requires pre-trained bicubic scale 2 or bicubic multiscale model. Here, we assume that you already downloaded the model bicubic_x2.t7 and bicubic_multiscale.t7 in the NTIRE2017/demo/model directory. It is possible to start the traning from scratch by removing -preTrained option in the script.

   cd $makeReposit/NTIRE2017/code
   
   sh training.sh