Physics-driven deep learning enables temporal compressive coherent diffraction imaging

Chen, Ziyang; Zheng, Siming; Tong, Zhishen; Yuan, Xin

doi:10.1364/optica.454582

Cited by 25 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to spectral SCI reconstruction as shown in this work, we do believe our network can be used in medical images [ 59 ], image compression [ 60 ], temporal compressive coherent diffraction imaging [ 61 ], and video compressive sensing [ 62 , 63 , 64 , 65 , 66 ].…”

Section: Discussionmentioning

confidence: 99%

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

Zheng

Zhu

Chen

2023

Entropy

Self Cite

View full text Add to dashboard Cite

In order to capture the spatial-spectral (x,y,λ) information of the scene, various techniques have been proposed. Different from the widely used scanning-based methods, spectral snapshot compressive imaging (SCI) utilizes the idea of compressive sensing to compressively capture the 3D spatial-spectral data-cube in a single-shot 2D measurement and thus it is efficient, enjoying the advantages of high-speed and low bandwidth. However, the reconstruction process, , to retrieve the 3D cube from the 2D measurement, is an ill-posed problem and it is challenging to reconstruct high quality images. Previous works usually use 2D convolutions and preliminary attention to address this challenge. However, these networks and attention do not exactly extract spectral features. On the other hand, 3D convolutions can extract more features in a 3D cube, but increase computational cost significantly. To balance this trade-off, in this paper, we propose a hybrid multi-dimensional attention U-Net (HMDAU-Net) to reconstruct hyperspectral images from the 2D measurement in an end-to-end manner. HMDAU-Net integrates 3D and 2D convolutions in an encoder–decoder structure to fully utilize the abundant spectral information of hyperspectral images with a trade-off between performance and computational cost. Furthermore, attention gates are employed to highlight salient features and suppress the noise carried by the skip connections. Our proposed HMDAU-Net achieves superior performance over previous state-of-the-art reconstruction algorithms.

show abstract

Section: Discussionmentioning

confidence: 99%

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

Zheng

Zhu

Chen

2023

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…TCSRM provides a powerful tool for the observation of high-speed dynamics of fine structures, especially in hydromechanics and biomedical fields, such as microflow velocity measurement, 34 organelle interactions, 35 intracellular transports, 36 and neural dynamics 37 . In addition, the framework of TCSRM can also offer guidance for achieving higher imaging speed and spatial resolution in holography, 38 coherent diffraction imaging, 39 and fringe projection profilometry 40 …”

Section: Discussionmentioning

confidence: 99%

Temporal compressive super-resolution microscopy at frame rate of 1200 frames per second and spatial resolution of 100 nm

Yao

et al. 2023

Adv. Photon.

View full text Add to dashboard Cite

.Various super-resolution microscopy techniques have been presented to explore fine structures of biological specimens. However, the super-resolution capability is often achieved at the expense of reducing imaging speed by either point scanning or multiframe computation. The contradiction between spatial resolution and imaging speed seriously hampers the observation of high-speed dynamics of fine structures. To overcome this contradiction, here we propose and demonstrate a temporal compressive super-resolution microscopy (TCSRM) technique. This technique is to merge an enhanced temporal compressive microscopy and a deep-learning-based super-resolution image reconstruction, where the enhanced temporal compressive microscopy is utilized to improve the imaging speed, and the deep-learning-based super-resolution image reconstruction is used to realize the resolution enhancement. The high-speed super-resolution imaging ability of TCSRM with a frame rate of 1200 frames per second (fps) and spatial resolution of 100 nm is experimentally demonstrated by capturing the flowing fluorescent beads in microfluidic chip. Given the outstanding imaging performance with high-speed super-resolution, TCSRM provides a desired tool for the studies of high-speed dynamical behaviors in fine structures, especially in the biomedical field.

show abstract

“…Optical signal processing based on linear transform always belongs to a unitary transform, intelligent weights should be enforced on unitary constraints in the training [28][29]. Commonly, Euclidean gradient in CVNN guides the optimal descend direction without any weights constraints on the high-dimensional smooth surface, the deepest gradient descend could be implemented to generate resulting direction of weights-update until iterative convergences.…”

Section: Unitary Learning In Conditional Modesmentioning

confidence: 99%

“…Commonly, Euclidean gradient in CVNN guides the optimal descend direction without any weights constraints on the high-dimensional smooth surface, the deepest gradient descend could be implemented to generate resulting direction of weights-update until iterative convergences. Lagrange multipliers methods always may be used for extra constraints on unitary variables that penalize the loss function [28]. However, it is very difficult to implement particularly for deep neural networks because the unitary constraints are so many enforced on each layer that computational consumptions are too heavy.…”

Section: Unitary Learning In Conditional Modesmentioning

confidence: 99%

Unitary learning in conditional modes for deep optics neural networks

Zhang

Xiao

Zhong

2023

Conference on Infrared, Millimeter, Terahertz Waves and Applications (IMT2022)

View full text Add to dashboard Cite

Complex-valued backprogation (CVBP) is an essential algorithm parsing a large amount of complex data for statistical inferences to deep optics neural networks. However, the operating optical transformations are almost unitary, which require the unitary backpropagation algorithms to meet the hetero-equivalence from their mathematical to optical mechanisms. This paper mainly compares two compact formulations for a common CVBP and its unitary-variants under holomorphic and compatible conditions, for which composite mechanisms are invented. For unitary weights-update, essentially, the Riemannian gradient of a CVBP is expanded from Euclidean gradient by executing an exponential mapping onto its unitary manifold. The convergent speed and accuracy, as well as the evolutions of their energies and averaging-phases at each layer are progressively investigated and compared, which reflect the dynamic behaviors on the network convergences. We find that not all the nonlinear activations under the respective conditions are always ensured to convergences, and the averaging-phases still remain several jumps although the energies at current layer situate at the convergent districts.

show abstract

Physics-driven deep learning enables temporal compressive coherent diffraction imaging

Cited by 25 publications

References 20 publications

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

Temporal compressive super-resolution microscopy at frame rate of 1200 frames per second and spatial resolution of 100 nm

Unitary learning in conditional modes for deep optics neural networks

Contact Info

Product

Resources

About