Multispectral filter array and demosaicking for pathological images

Abstract

A capturing system with multispectral filter array (MSFA) technology is proposed for shortening the capture time and reducing costs. Therein, a mosaicked image captured using an MSFA is demosaicked to reconstruct multispectral images (MSIs). Joint optimization of the spectral sensitivity of the MSFAs and demosaicking is considered, and pathology-specific multispectral imaging is proposed. This optimizes the MSFA and the demosaicking matrix by minimizing the reconstruction error between the training data of a hematoxylin and eosin-stained pathological tissue and a demosaicked MSI using a cost function. Initially, the spectral sensitivity of the filter array is set randomly and the mosaicked image is obtained from the training data. Subsequently, a reconstructed image is obtained using Wiener estimation. To minimize the reconstruction error, the spectral sensitivity of the filter array and the Wiener estimation matrix are optimized iteratively through an interior-point approach. The effectiveness of the proposed MSFA and demosaicking is demonstrated by comparing the recovered spectrum and RGB image with those obtained using a conventional method.

Illumination-robust multispectral demosaicing

“…These methods respectively use the edge and high-frequency information of the estimated PPI to interpolate the missing levels of each channel. Shinoda et al [21] propose a multispectral local directional interpolation (MLDI) that combines BTES edge-sensing mechanism with SD spectral correlation implementation.…”

Snapshot multispectral polarization imaging using a photonic crystal filter array

“…They also proposed a linear modulation demosaicking method for the FSFA. This paper has been submitted to International Workshop on Image Sensors and Imaging Systems (IWISS2018) (Invited talk) Beside these approaches, we proposed a joint optimizing method for the spectral sensitivity of an MSFA and the demosaicking with using a training data, and pathology-specific multispectral imaging is proposed [21]. When full-resolution MSIs, which are not mosaicked, are employed, the MSFA pattern and the demosaicking matrix can be optimized by minimizing the error between the original and demosaicked images during a simulation, as shown in Fig.…”

“…However, it requires the demosaicking process to recover a multispectral polarization image from the captured grayscale image because various spectral and polarization components are mixed in one pixel. The capturing process is considered as a linear model and various images are recovered by solving the inverse problem of the linear model as well as [21]. We attached the fabricated filter array to a monochrome imager (ICX205AL, SONY) by ultraviolet curing.…”

“…This recovering process is referred to as demosaicking. Various filter array patterns and demosaicking methods [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21] have been proposed for improving demosaicked image quality. Because the number of non-measured pixels in an MSFA is larger than that in an RGB Bayer color filter array, a more appropriate MSFA pattern and an accurate demosaicking method are required.…”

N-LMMSE Demosaicing for Spectral Filter Arrays

“…51,52 Compressive sensing to compensate for the increasing sparsity have also been used successfully. [53][54][55] Specific to technology, such as Fourier SFA 56,57 and hybrid multi-camera approaches 58 and specific to application demosaicing 59 have also been developed.…”