Deep Joint Design of Color Filter Arrays and Demosaicing

Henz, Bernardo; Gastal, Eduardo S. L.; Oliveira, Manuel M.

doi:10.1111/cgf.13370

Cited by 45 publications

(46 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning-based CFA Design: By minimizing the average error on a color dataset, Lu and Vetterli [22] used an iterative algorithm to solve a least squares CFA design problem. Chakrabarti [34] and Henz et al [35] proposed to learn the optimal CFA pattern by using a deep neural network.…”

Section: Related Workmentioning

confidence: 99%

Color Filter Arrays for Quanta Image Sensors

Elgendy¹,

Chan

2020

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Quanta image sensor (QIS) is envisioned to be the next generation image sensor after CCD and CMOS. In this paper, we discuss how to design color filter arrays for QIS and other small pixels. Designing color filter arrays for small pixels is challenging because maximizing the light efficiency while suppressing aliasing and crosstalk are conflicting tasks. We present an optimization-based framework which unifies several mainstream color filter array design methodologies. Our method offers greater generality and flexibility. Compared to existing methods, the new framework can simultaneously handle luminance sensitivity, chrominance sensitivity, cross-talk, antialiasing, manufacturability and orthogonality. Extensive experimental comparisons demonstrate the effectiveness of the framework. Spatial CFA Design: By suppressing the Moiré artifacts and crosstalk while keeping the demosaicing algorithm simple, Lukac and Plataniotis [20] proposed a CFA and compared it with other CFAs using a universal demosaicking method. However, their work did not provide a mathematical framework to analyze the CFA optimality. AbstractThis supplementary report provides the following additional information of the main article.

show abstract

Section: Related Workmentioning

confidence: 99%

Color Filter Arrays for Quanta Image Sensors

Elgendy¹,

Chan

2020

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…A very common approach is bilinear interpolation, as well as, other variants of this method which are adaptive to image edges [18,30]. During the last years, the image demosaicking task witnessed an incredible quantitative and qualitative performance increase via the use of neural network approaches like those in [10,17] and most recently in [21]. This performance increase holds true even under the presence of noise perturbing the camera sensor readings.…”

Section: Image Demosaickingmentioning

confidence: 99%

Iterative Residual CNNs for Burst Photography Applications

Kokkinos

Lefkimmiatis

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Modern inexpensive imaging sensors suffer from inherent hardware constraints which often result in captured images of poor quality. Among the most common ways to deal with such limitations is to rely on burst photography, which nowadays acts as the backbone of all modern smartphone imaging applications. In this work, we focus on the fact that every frame of a burst sequence can be accurately described by a forward (physical) model. This, in turn, allows us to restore a single image of higher quality from a sequence of low-quality images as the solution of an optimization problem. Inspired by an extension of the gradient descent method that can handle non-smooth functions, namely the proximal gradient descent, and modern deep learning techniques, we propose a convolutional iterative network with a transparent architecture. Our network uses a burst of lowquality image frames and is able to produce an output of higher image quality recovering fine details which are not distinguishable in any of the original burst frames. We focus both on the burst photography pipeline as a whole, i.e., burst demosaicking and denoising, as well as on the traditional Gaussian denoising task. The developed method demonstrates consistent state-of-the art performance across the two tasks and as opposed to other recent deep learning approaches does not have any inherent restrictions either to the number of frames or their ordering.

show abstract

“…The overall design takes all of the above factors into consideration. In the area of color imaging, Henz et al [40] proposed an overall color filter array optimization method using deep convolutional neural networks. Similarly, Nie et al [41] jointly optimized filters and recovered spectral images by optimizing the weights in CNN.…”

Section: Related Workmentioning

confidence: 99%

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Xie

et al. 2019

Sensors

View full text Add to dashboard Cite

Multispectral filter array (MSFA)-based imaging is a compact, practical technique for snapshot spectral image capturing and reconstruction. The imaging and reconstruction quality is highly influenced by the spectral sensitivities and spatial arrangement of channels on MSFAs, and the used reconstruction method. In order to design a MSFA with high imaging capacity, we propose a sparse representation based approach to optimize spectral sensitivities and spatial arrangement of MSFAs. The proposed approach first overall models the various errors associated with spectral reconstruction, and then uses a global heuristic searching method to optimize MSFAs via minimizing the estimated error of MSFAs. Our MSFA optimization method can select filters from off-the-shelf candidate filter sets while assigning the selected filters to the designed MSFA. Experimental results on three datasets show that the proposed method is more efficient, flexible, and can design MSFAs with lower spectral construction errors when compared with existing state-of-the-art methods. The MSFAs designed by our method show better performance than others even using different spectral reconstruction methods.

show abstract

Deep Joint Design of Color Filter Arrays and Demosaicing

Cited by 45 publications

References 35 publications

Color Filter Arrays for Quanta Image Sensors

Color Filter Arrays for Quanta Image Sensors

Iterative Residual CNNs for Burst Photography Applications

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Contact Info

Product

Resources

About