D$^\text{2}$UF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion

Jácome, Román; Bacca, Jorge; Argüello, Henry

doi:10.48550/arxiv.2205.12158

Cited by 2 publications

(2 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, autoencoders, residual-networks, or the well-known UNet, have been used for image recovery. Furthermore, unrolledbased approaches exploit iterative reconstruction algorithms as layers [26]. More recently, DNN with self-attention mechanisms along spatial or spectral dimensions have also been proposed [27].…”

Section: B Computational Decodermentioning

confidence: 99%

Deep Optical Coding Design in Computational Imaging: A data-driven framework

Argüello

Bacca

Kariyawasam

et al. 2023

IEEE Signal Process. Mag.

View full text Add to dashboard Cite

Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set the distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder. Specifically, by modeling the COI measurements through a fully differentiable image formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the computational decoder can be optimized in an end-to-end (E2E) manner. Moreover, by optimizing just CEs in the same framework, inference tasks can be performed from pure optics. This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework. Since the E2E framework can handle different inference applications by changing the loss function and the DNN, we present low-level tasks such as spectral imaging reconstruction or high-level tasks such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures. Finally, we illustrate classification and 3D object recognition applications performed at the speed of the light using all-optics DNN. I. INTRODUCTIONWe are in a new era of computational optical imaging (COI) systems that break traditional sensing limits leveraged by computational signal processing. COI systems have enabled the acquisition of high-dimensional information of the scene such as polarization [1], spectrum [2], depth [3], time [4] with unprecedented performance in commercial applications such as medical imaging, precision agriculture, surveillance, and vision-guided vehicles [5]. With current measurement devices it is only possible to acquire low-dimensional intensity values of high-dimensional scenes.Therefore, the main goal of computational imaging design is to encode the desired information in a low-dimensional

show abstract

Section: B Computational Decodermentioning

confidence: 99%

Deep Optical Coding Design in Computational Imaging: A data-driven framework

Argüello

Bacca

Kariyawasam

et al. 2023

IEEE Signal Process. Mag.

View full text Add to dashboard Cite

show abstract

“…Consequently, the parameter ρ plays an essential role in the optimal performance and coded aperture implementability. This work uses an exponential increase strategy [20,26], which consist of starting with low values of ρ at the first epochs to obtain the direction of the desired task values, and then ρ is increased to guarantee binary values.…”

Section: End-to-end Optimizationmentioning

confidence: 99%

Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification

Bacca¹,

Hernandez-Rojas²,

Argüello³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Compressive spectral imaging (CSI) has emerged as an attractive compression and sensing technique, primarily to sense spectral regions where traditional systems result in highly costly such as in the near-infrared spectrum. Recently, it has been shown that spectral classification can be performed directly in the compressive domain, considering the amount of spectral information embedded in the measurements, skipping the reconstruction step. Consequently, the classification quality directly depends on the set of coding patterns employed in the sensing step. Therefore, this work proposes an end-to-end approach to jointly design the coding patterns used in CSI and the network parameters to perform spectral classification directly from the embedded near-infrared compressive measurements. Extensive simulation on the three-dimensional coded aperture snapshot spectral imaging (3D-CASSI) system validates that the proposed design outperforms traditional and random design in up to 10% of classification accuracy.

show abstract

D$^\text{2}$UF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion

Cited by 2 publications

References 49 publications

Deep Optical Coding Design in Computational Imaging: A data-driven framework

Deep Optical Coding Design in Computational Imaging: A data-driven framework

Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification

Contact Info

Product

Resources

About