For paper, slides and poster, please refer to our project page

We propose a simple, interpretable framework for solving a wide range of image reconstruction problems such as denoising and deconvolution. Given a corrupted input image, the model synthesizes a spatially varying linear filter which, when applied to the input image, reconstructs the desired output. The model parameters are learned using supervised or self-supervised training. We test this model on three tasks: non-uniform motion blur removal, lossy-compression artifact reduction and single image super resolution. We demonstrate that our model substantially outperforms state-of-the-art methods on all these tasks and is significantly faster than optimization-based approaches to deconvolution. Unlike models that directly predict output pixel values, the predicted filter flow is controllable and interpretable, which we demonstrate by visualizing the space of predicted filters for different tasks.

keywords: inverse problem, spatially-variant blind deconvolution, low-level vision, non-uniform motion blur removal, compression artifact reduction, single image super-resolution, filter flow, interpretable model, per-pixel twist, self-supervised learning, image distribution learning.

The jupyter script provided here is self-contained. Please run task01_deblur.ipynb directly to see how our model performs for non-uniform motion blur removal. Please go to this folder to have a quick look at more visualizations! Besides, more demos are on the way.

If you find anything provided here inspires you, please cite our arxiv paper (hig-resolution draft pdf, 44Mb):

@inproceedings{kong2018PPF,
  title={Image Reconstruction with Predictive Filter Flow},
  author={Kong, Shu and Fowlkes, Charless},
  booktitle={arxiv},
  year={2018}
}

last update: 10/28/2018

Shu Kong

issues/questions addressed here: aimerykong At g-m-a-i-l dot com