Deeply learned broadband encoding stochastic hyperspectral imaging

Zhang, Wenyi; Song, Hongya; He, Xin; Huang, Longqian; Zhang, Xiyue; Zheng, Junyan; Shen, Weidong; Hao, Xiang; Liu, Xü

doi:10.1038/s41377-021-00545-2

Cited by 84 publications

(55 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with conventional reconstruction algorithms using linear matrix calculation-based compressed sensing, 13,19 deep learning-based reconstruction algorithm leads to prominent improvements in both reconstruction speed and denoising ability. 28 For the former aspect, time reduction is not obvious in single-pixel spectral reconstruction. However, when applied in spectral imaging with millions of pixels, DNN provides a much higher reconstruction speed benefiting from parallel computing.…”

Section: ■ Results and Discussionmentioning

confidence: 99%

Deep Learning-Based Miniaturized All-Dielectric Ultracompact Film Spectrometer

Wen

Gao

et al. 2022

ACS Photonics

Self Cite

View full text Add to dashboard Cite

Conventional benchtop spectrometers with bulky dispersive optics and long optical path lengths display limitations where the significance of miniaturization, real-time detection, and low cost transcend the ultrafine resolution and wide spectral range. Here, we demonstrate a miniaturized all-dielectric ultracompact film spectrometer based on deep learning working in the single-shot mode. The scheme employs 16 spectral encoders with simple five-layer film stacks where merely the thickness of the intermediate high-index modulation layer is varied to realize unique encoded transmission spectra. Structural parameters as well as transmission spectra of the filters are predesigned to guarantee weak correlation and highly efficient encoding. Leveraging a trained reconstruction network, the absolute spectra of various nonluminous samples are successfully reconstructed excluding the emitting spectrum of the light source and the spectral response of the detector. The remarkable reconstructed spectral imaging result for the color board is presented and the reconstructed spectra match well with the measured ones for different patches using the identical network. We utilized the least number of spectral encoders ever since to guarantee efficient encoding, along with the single thickness-variant modulation layer, which shows potential for mass, rapid, large-area production by combining deposition with nanoimprint. Instead of the synthetic Gaussian line shape spectra, a training dataset composed of diverse spectrum types is adopted to achieve fine generalization of the trained reconstruction network. In addition, by retraining the neural network, the reconstruction network is modified to fit for the actual filter functions of the spectral encoders, thus better reconstruction performance. The proposed miniaturized spectrometer has great prospects in the fields of consumer electronics, environmental monitoring, and disaster prevention.

show abstract

Section: ■ Results and Discussionmentioning

confidence: 99%

Deep Learning-Based Miniaturized All-Dielectric Ultracompact Film Spectrometer

Wen

Gao

et al. 2022

ACS Photonics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The subsequent computational reconstruction is performed in MATLAB and takes about 300 ms on a desktop computer with an Intel i5-8500 CPU and 32 GB RAM. The speed and accuracy of the spectropolarimetric reconstruction may be further improved through the use of data-driven deep learning-based algorithms [29,40].…”

Section: Experimental Demonstrationmentioning

confidence: 99%

“…In recent years, metasurface-based polarimeters and spectrometers have also been developed. Besides conventional approaches based on either division-of-amplitude [18][19][20][21][22][23] or division-of-time [24,25], computational polarimeters and spectrometers have recently been demonstrated, in which the polarization or spectrum of the incident light can be encoded using a tunable grapheneintegrated metasurface [26] or a metasurface array [27][28][29][30], and decoded through computational reconstruction. However, the simultaneous multiplexing and reconstruction of multi-dimensional light field information remain a challenging task.…”

Section: Introductionmentioning

confidence: 99%

Computational spectropolarimetry with a tunable liquid crystal metasurface

Chen

Wen

et al. 2022

eLight

View full text Add to dashboard Cite

While conventional photodetectors can only measure light intensity, the vectorial light field contains much richer information, including polarization and spectrum, that are essential for numerous applications ranging from imaging to telecommunication. However, the simultaneous measurement of multi-dimensional light field information typically requires the multiplexing of dispersive or polarization-selective elements, leading to excessive system complexity. Here, we demonstrate a near-infrared spectropolarimeter based on an electrically-tunable liquid crystal metasurface. The tunable metasurface, which acts as an encoder of the vectorial light field, is tailored to support high-quality-factor guided-mode resonances with diverse and anisotropic spectral features, thus allowing the full Stokes parameters and the spectrum of the incident light to be computationally reconstructed with high fidelity. The concept of using a tunable metasurface for multi-dimensional light field encoding may open up new horizons for developing vectorial light field sensors with minimized size, weight, cost, and complexity.

show abstract

“…The network is like an autoencoder, where the input HSI is downsampled and then reconstructed by a decoder network. Zhang et al [62] designed and fabricated a broadband encoding stochastic camera containing 16 trainable projectors that map high-dimensional spectra to lower-dimensional intensity matrices. Recently, Liutao et al [56] proposed FS-Net, a filter-selection network for task-specific hyperspectral image analysis.…”

Section: Reconstruction By Sparse Coding and Deep Learningmentioning

confidence: 99%

Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders

Makarenko¹,

Burguete-Lopez²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Hyperspectral imaging has attracted significant attention to identify spectral signatures for image classification and automated pattern recognition in computer vision. State-ofthe-art implementations of snapshot hyperspectral imaging rely on bulky, non-integrated, and expensive optical elements, including lenses, spectrometers, and filters. These macroscopic components do not allow fast data processing for, e.g. real-time and high-resolution videos. This work introduces Hyplex™, a new integrated architecture addressing the limitations discussed above. Hyplex™ is a CMOS-compatible, fast hyperspectral camera that replaces bulk optics with nanoscale metasurfaces inversely designed through artificial intelligence. Hyplex™ does not require spectrometers but makes use of conventional monochrome cameras, opening up the possibility for real-time and high-resolution hyperspectral imaging at inexpensive costs. Hyplex™ exploits a modeldriven optimization, which connects the physical metasurfaces layer with modern visual computing approaches based on end-to-end training. We design and implement a prototype version of Hyplex™ and compare its performance against the state-of-the-art for typical imaging tasks such as spectral reconstruction and semantic segmentation. In all benchmarks, Hyplex™ reports the smallest reconstruction error. We additionally present what is, to the best of our knowledge, the largest publicly available labeled hyperspectral dataset for semantic segmentation. 1

show abstract

Deeply learned broadband encoding stochastic hyperspectral imaging

Cited by 84 publications

References 24 publications

Deep Learning-Based Miniaturized All-Dielectric Ultracompact Film Spectrometer

Deep Learning-Based Miniaturized All-Dielectric Ultracompact Film Spectrometer

Computational spectropolarimetry with a tunable liquid crystal metasurface

Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders

Contact Info

Product

Resources

About