Learning Spatially-Variant MAP Models for Non-blind Image Deblurring

Abstract

The classical maximum a-posteriori (MAP) framework for non-blind image deblurring requires defining suitable data and regularization terms, whose interplay yields the desired clear image through optimization. The vast majority of prior work focuses on advancing one of these two crucial ingredients, while keeping the other one standard. Considering the indispensable roles and interplay of both data and regularization terms, we propose a simple and effective approach to jointly learn these two terms, embedding deep neural networks within the constraints of the MAP framework, trained in an end-to-end manner. The neural networks not only yield suitable image-adaptive features for both terms, but actually predict per-pixel spatially-variant features instead of the commonly used spatially-uniform ones. The resulting spatially-variant data and regularization terms particularly improve the restoration of fine-scale structures and detail. Quantitative and qualitative results underline the effectiveness of our approach, substantially outperforming the current state of the art.

DeepRLS: A Recurrent Network Architecture with Least Squares Implicit Layers for Non-blind Image Deconvolution

“…From the reported results it is clear, that our proposed network outperforms all competing methods by a good margin in both color and grayscale deblurring on small and large noise levels. In particular, for the case of color deblurring our method leads to improved results of almost 0.3dB for both 1% and 5% Gaussian noise compared to the recent best performing network [14]. The improved performance is even more pronounced in the case of grayscale image deblurring, where our method outperforms the second best FDN network [29] by nearly 0.4dB in all tested cases.…”

“…To show the general applicability of our network, we have conducted a separate experiment considering deblur-ring of saturated images. For this purpose, we have used a similar procedure as the one proposed in [14] to obtain train samples. Although the degradation model for such cases departs from the adopted model of Eq.…”

“…To validate all the methods, we follow the protocol proposed by [47] and we compute all the metrics by considering the central part of the original image (we discard 50 pixel from each border), so as to avoid the influence of boundary artifacts in the SSIM and PSNR scores. Several synthetic benchmarks were previously proposed to evaluate deblurring methods on large set of images with saturated pixels [14]. Unfortunately, none of them contain neither publicly released data, nor concrete instructions on how to reproduce them.…”

“…Image deconvolution is a very challenging problem and a plethora of methods have been proposed in the literature to address it. We can classify all these methods into two categories, those that deal with the estimation of both the underlying sharp image and the blur kernel (psf) [9,12,42,49,54] and those that work under the assumption that the blur kernel is given and aim to estimate only the underlying image [13,14,20,29,55]. The methods in the first class are referred to as blind deconvolution methods while those in the second as non-blind deconvolution methods.…”

“…Then, the image deconvolution is re-casted to a constrained optimization problem. However, the most recent paradigm, which has shown great potentials, involves deep-learning methods which are able both to implicitly encode prior image information and obtain the deblurred result relying on specific network architectures [13,14,29,42].…”