malav5372 / supernovasr Goto Github PK

I bring to you SupernovaSR based on this research paper. This project was born out of a pressing challenge within the realm of Single Image Super-Resolution (SISR). Existing methods often grapple with inadequate performance when their assumed degradation models fail to account for the intricate complexities of real-world images.

By introducing a novel and practical degradation model, SupernovaSR aims to significantly improve the quality and practicality of deep Super-Resolution techniques, ultimately redefining the landscape of image enhancement and restoration.

Feel free to explore the codebase, experiment with the model, and contribute to the advancement of Single Image Super-Resolution with SupernovaSR.

1. Train SupernovaSRNet
   1. Modify train_SupernovaSR_x4_psnr.json e.g., "gpu_ids": [0], "dataloader_batch_size": 4
   2. Training with DataParallel

   python main_train_psnr.py --opt options/train_SupernovaSR_x4_psnr.json

   2. Training with DistributedDataParallel - 4 GPUs

   python -m torch.distributed.launch --nproc_per_node=4 --master_port=1234 main_train_psnr.py --opt options/train_SupernovaSR_x4_psnr.json  --dist True

2. Train SupernovaSR
   1. Put SupernovaSRNet model (e.g., '400000_G.pth') into superresolution/SupernovaSR_x4_gan/models
   2. Modify train_SupernovaSR_x4_gan.json e.g., "gpu_ids": [0], "dataloader_batch_size": 4
   3. Training with DataParallel

   python main_train_gan.py --opt options/train_SupernovaSR_x4_gan.json

   3. Training with DistributedDataParallel - 4 GPUs

   python -m torch.distributed.launch --nproc_per_node=4 --master_port=1234 main_train_gan.py --opt options/train_SupernovaSR_x4_gan.json  --dist True

3. Test SupernovaSR model 'xxxxxx_E.pth' by modified main_test_SupernovaSR.py
   1. 'xxxxxx_E.pth' is more stable than 'xxxxxx_G.pth'

✨ Some visual examples: oldphoto2; building; tiger; pattern; oldphoto6; comic_01; comic_03; comic_04

Comparison:

• Testing code
• Main idea
• Comparison
• More visual results on RealSRSet dataset

• main_test_SupernovaSR.py
• model_zoo (_Download the models from Google drive
  • Proposed:
    • SupernovaSR.pth [Google drive]
    • SupernovaSRNet.pth [Google drive]

Design a new degradation model to synthesize LR images for training:

• 1) Make the blur, downsampling and noise more practical
  • Blur: two convolutions with isotropic and anisotropic Gaussian kernels from both the HR space and LR space
  • Downsampling: nearest, bilinear, bicubic, down-up-sampling
  • Noise: Gaussian noise, JPEG compression noise, processed camera sensor noise
• 2) Degradation shuffle: instead of using the commonly-used blur/downsampling/noise-addition pipeline, we perform randomly shuffled degradations to synthesize LR images

Some notes on the proposed degradation model:

• The degradation model is mainly designed to synthesize degraded LR images. Its most direct application is to train a deep blind super-resolver with paired LR/HR images. In particular, the degradation model can be performed on a large dataset of HR images to produce unlimited perfectly aligned training images, which typically do not suffer from the limited data issue of laboriously collected paired data and the misalignment issue of unpaired training data.

• The degradation model tends to be unsuited to model a degraded LR image as it involves too many degradation parameters and also adopts a random shuffle strategy.

• The degradation model can produce some degradation cases that rarely happen in real-world scenarios, while this can still be expected to improve the generalization ability of the trained deep blind super-resolver.

• A DNN with large capacity has the ability to handle different degradations via a single model. This has been validated multiple times. For example, DnCNN is able to handle SISR with different scale factors, JPEG compression deblocking with different quality factors and denoising for a wide range of noise levels, while still having a performance comparable to VDSR for SISR. It is worth noting that even when the super-resolver reduces the performance for unrealistic bicubic downsampling, it is still a preferred choice for real SISR.

• One can conveniently modify the degradation model by changing the degradation parameter settings and adding more reasonable degradation types to improve the practicability for a certain application.

Left: real images | Right: super-resolved images with scale factor 4