Snapshot multispectral endomicroscopy

Meng, Ziyi; Qiao, Mu; Ma, Jiawei; Yu, Zhenming; Xu, Kun; Yuan, Xin

doi:10.1364/ol.393213

Cited by 74 publications

(29 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With this simple change and keeping the network structure, significant improvement has been achieved on both simulation and real data. Our proposed method can also be used in other spectral imaging systems [28,29] as well as other snapshot compressive systems such as video [5,30], spectral-temporal [4], spatialtemporal [6] imaging.…”

Section: Discussionmentioning

confidence: 99%

Perception Inspired Deep Neural Networks For Spectral Snapshot Compressive Imaging

Meng

Yuan

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

We consider the inverse problem of coded aperture snapshot spectral imaging (CASSI), which captures the spatio-spectral data-cube using a snapshot 2D measurement and reconstructs the 3D hyperspectral images using algorithms. Recent advances of deep learning have boosted the image quality of the reconstructed hyperspectral images significantly, and this leads to an end-to-end real-time capture and reconstruction system. However, the network design for CASSI reconstruction is still at the incubation stage and usually an off-the-shelf network is employed and re-purposed. In this work, from a different perspective, inspired by the fact that most existing hyperspectral images are still in the visible bandwidth, we introduce the perceptual loss into the deep neural network for CASSI reconstruction. Extensive results on both simulation and real data demonstrate that with this small change, the reconstructed image quality can be improved dramatically using the same network.

show abstract

Section: Discussionmentioning

confidence: 99%

Perception Inspired Deep Neural Networks For Spectral Snapshot Compressive Imaging

Meng

Yuan

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Beyond automatic diagnosis and super‐resolution approaches in IVM, recent advances also highlight ways in which DL can enable novel instrumentation development and image reconstructions to enable new functionalities for compact microscopy systems. Such examples include multispectral endomicroscopy [129], more robust mosaicking for FOV expansion [130], and end‐to‐end image reconstruction using disordered fiber‐optic probes [131,132]. We anticipate that similarly to ex vivo microscopy, in the coming years DL will be increasingly utilized to overcome physical constraints, augment contrast mechanisms, and enable new capabilities for IVM systems.…”

Section: Applications In Biomedical Opticsmentioning

confidence: 99%

Deep Learning in Biomedical Optics

et al. 2021

View full text Add to dashboard Cite

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC

show abstract

“…Equipped with rich spectral information, hyperspectral imaging has wide applications in agriculture, food detection, mineral detection and medical imaging [82].…”

Section: Hyperspectral Imagingmentioning

confidence: 99%

“…Use deep learning to decode the high-dimensional data from multiple encoded measurements [192] [193] Propose a deep neural network based on a standard tensor ADMM algorithm [175] Build a video CI system using a digital micro-mirror device and develop end-to-end convolutional neural networks for reconstruction [176] Build a multispectral endomicroscopy CI system using coded aperture plus disperser and develop end-to-end convolutional neural networks for reconstruction [82] Propose a new auto-focus mechanisms based on reinforcement learning [195] AI optimizes structure and design of CI system Assist in designing the physical layout of imaging system for compact imaging [201] Perform end-to-end optimization of an optical system [48] [88] [202]- [205] Introduce special optical elements, or replace the original ones for lightweight system [206]- [209] Show better results by novel end-to-end network frameworks via optimization [191] [210]- [213] AI promotes scene adaptive CI system…”

Section: Ai Improves Quality and Efficiency Of CI Systemmentioning

confidence: 99%

Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision

Suo¹,

Zhang²,

Gong³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Signal capture stands in the forefront to perceive and understand the environment and thus imaging plays the pivotal role in mobile vision. Recent explosive progresses in Artificial Intelligence (AI) have shown great potential to develop advanced mobile platforms with new imaging devices. Traditional imaging systems based on the "capturing images first and processing afterwards" mechanism cannot meet this unprecedented demand. Differently, Computational Imaging (CI) systems are designed to capture high-dimensional data in an encoded manner to provide more information for mobile vision systems. Thanks to AI, CI can now be used in real systems by integrating deep learning algorithms into the mobile vision platform to achieve the closed loop of intelligent acquisition, processing and decision making, thus leading to the next revolution of mobile vision. Starting from the history of mobile vision using digital cameras, this work first introduces the advances of CI in diverse applications and then conducts a comprehensive review of current research topics combining CI and AI. Motivated by the fact that most existing studies only loosely connect CI and AI (usually using AI to improve the performance of CI and only limited works have deeply connected them), in this work, we propose a framework to deeply integrate CI and AI by using the example of self-driving vehicles with high-speed communication, edge computing and traffic planning. Finally, we outlook the future of CI plus AI by investigating new materials, brain science and new computing techniques to shed light on new directions of mobile vision systems.

show abstract

Snapshot multispectral endomicroscopy

Cited by 74 publications

References 22 publications

Perception Inspired Deep Neural Networks For Spectral Snapshot Compressive Imaging

Perception Inspired Deep Neural Networks For Spectral Snapshot Compressive Imaging

Deep Learning in Biomedical Optics

Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision

Contact Info

Product

Resources

About