Low-complexity linear demosaicing using joint spatial-chromatic image statistics

Abstract

We present an efficient Linear Minimum Mean Square Error (LMMSE) method for reconstructing full color images from single sensor Color Filter Array (CFA) data. We use a representative set of full color images to estimate the joint spatial-chromatic covariance among pixel color components. Then, we derive from it a set of joint color-space, small linear kernels which predict the missing color samples as linear combinations of their neighbor observed samples. The color arrangement of the local mosaic varies with the window's location, and this results into a different predictor for every local mosaic and color sample. As an extension, we include blur and noise in the training process, obtaining localized mosaic-constrained Wiener estimators that partially compensate for these degradations. We show that this simple method provides an excellent trade-off between performance and computational cost.

Practical implementation of LMMSE demosaicing using luminance and chrominance spaces

“…With the assumption that the acquisition process is invariant over the image, which is widely used, it allows the design of space-invariant filters at this scale, or in other words of block shift-invariant filters [8]. In Portilla et al [9], the change over the basic pattern of the CFA is done by estimating four filters.…”

“…the size of matrix D i ) and to avoid any arbitrary truncation of the impulse response. A similar approach was recently used in [9] through the definition of a spatio-chromatic covariance matrices defined for the four elements of the superpixel.…”

“…They used a minimum mean square error approach with prior probability distributions on the image formation to design filters for demosaicing and deblurring the hyperspectral image acquired through a CFA-based camera. In Portilla et al [9] the filters were estimated from an image database and it is shown that LLMSE demosaicing can operate denoising and deblurring. In Hirakawa et al [35], a TLS denoising is designed conjointly with demosaicing and operated well on real noisy CMOS images.…”

Artificial Intelligence

“…Bilinear interpolation, for example, is very fast, but it processes independently each colour channel, thus ignoring the correlation between them. This results in poor results [143]. On the other hand, high performance iterative methods are too slow, and thus inadequate for real time image capture [144,145,146].…”

“…However, their good results allow having high-quality images if it is possible to post-process them after the capture. Finally, some existing methods are based on linear filtering taking into account the inter-channel correlation, then trying to reach a good balance between computation and image quality [147,148,143,149].…”