No abstract

Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.

Label-free multiphoton microscopy is a powerful platform for biomedical imaging. Recent advancements have demonstrated the capabilities of transient absorption microscopy (TAM) for label-free quantification of hemoglobin and stimulated Raman scattering (SRS) microscopy for pathological assessment of label-free virtual histochemical staining. We propose the combination of TAM and SRS with two-photon excited fluorescence (TPEF) to characterize, quantify, and compare hemodynamics, vessel structure, cell density, and cell identity in vivo between age groups. In this study, we construct a simultaneous nonlinear absorption, Raman, and fluorescence (SNARF) microscope with the highest reported in vivo imaging depth for SRS and TAM at 250–280 μm to enable these multimodal measurements. Using machine learning, we predict capillary-lining cell identities with 90% accuracy based on nuclear morphology and capillary relationship. The microscope and methodology outlined herein provides an exciting route to study several research topics, including neurovascular coupling, blood-brain barrier, and neurodegenerative diseases.