Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array

Abstract

We demonstrate the use of deep learning for fast spectral deconstruction of speckle patterns. The artificial neural network can be effectively trained using numerically constructed multispectral datasets taken from a measured spectral transmission matrix. Optimized neural networks trained on these datasets achieve reliable reconstruction of both discrete and continuous spectra from a monochromatic camera image. Deep learning is compared to analytical inversion methods as well as to a compressive sensing algorithm and shows favourable characteristics both in the oversampling and in the sparse undersampling (compressive) regimes. The deep learning approach offers significant advantages in robustness to drift or noise and in reconstruction speed. In a proof-of-principle demonstrator we achieve real time recovery of hyperspectral information using a multi-core, multi-mode fiber array as a random scattering medium.

Deep Learning Enabled Laser Speckle Wavemeter with a High Dynamic Range

“…[25] Additionally, harnessing the spectral characteristics of speckle, CNNs have found an application to achieve real-time recovery of hyperspectral information with a wavelength resolution of 5 nm. [26] In this study, we present a method based on deep learning and t-distributed stochastic neighbor embedding (t-SNE) [27] to classify and segment the speckle images corresponding to a given laser wavelength. An interesting aspect presented in this study is the automatic rejection of instrumental or environmental noise by the CNN, which enables a classification of speckle patterns with a wavelength precision of two attometres, representing a nine orders of magnitude improvement compared to previous studies with deep learning.…”

“…An interesting aspect presented in this study is the automatic rejection of instrumental or environmental noise by the CNN, which enables a classification of speckle patterns with a wavelength precision of two attometres, representing a nine orders of magnitude improvement compared to previous studies with deep learning. [26] This, coupled with the capability of the pre-trained CNN to segment the speckle images covering the entire visible spectrum, leads to a dynamic range improvement by six orders of magnitude. Going beyond the capability to identify the speckle-creating scatterer, [25] we additionally show that the trained CNN, in combination with t-SNE, can recognize the wavelength variations of speckle regardless of which scattering medium is used.…”

Analysis of non-iterative phase retrieval based on machine learning

“…The second approach enables faster non-iterative phase retrieval than the conventional methods, and it has been used for real-time diffraction imaging, imaging through scattering media, computer-generated holograms, wavefront sensing, and pulse measurement [27][28][29][30][31][32][33][34]. Also, such DNN-based inversion has been introduced to optical sensing methods other than phase retrieval [35][36][37][38].…”

Deep Learning-Based Image Classification through a Multimode Fiber in the Presence of Wavelength Drift

“…Recently, the advancements in computational means allow the realization of more complex neural network architectures that can handle data of high-dimensionality, bringing DNNs to the forefront of many leading technologies ranging from research and business to military and entertainment [23][24][25]. DNNs have shown remarkable capabilities in recovering information through MMFs for imaging applications, while preserving robustness against perturbations in the system [16][17][18]20,26].…”