SummaryBackgroundLiver fibrosis is the strongest histological risk factor for liver‐related complications and mortality in metabolic dysfunction‐associated fatty liver disease (MAFLD). Second harmonic generation/two‐photon excitation fluorescence (SHG/TPEF) is a powerful tool for label‐free two‐dimensional and three‐dimensional tissue visualisation that shows promise in liver fibrosis assessment.AimTo investigate combining multi‐photon microscopy (MPM) and deep learning techniques to develop and validate a new automated quantitative histological classification tool, named AutoFibroNet (Automated Liver Fibrosis Grading Network), for accurately staging liver fibrosis in MAFLD.MethodsAutoFibroNet was developed in a training cohort that consisted of 203 Chinese adults with biopsy‐confirmed MAFLD. Three deep learning models (VGG16, ResNet34, and MobileNet V3) were used to train pre‐processed images and test data sets. Multi‐layer perceptrons were used to fuse data (deep learning features, clinical features, and manual features) to build a joint model. This model was then validated in two further independent cohorts.ResultsAutoFibroNet showed good discrimination in the training set. For F0, F1, F2 and F3‐4 fibrosis stages, the area under the receiver operating characteristic curves (AUROC) of AutoFibroNet were 1.00, 0.99, 0.98 and 0.98. The AUROCs of F0, F1, F2 and F3‐4 fibrosis stages for AutoFibroNet in the two validation cohorts were 0.99, 0.83, 0.80 and 0.90 and 1.00, 0.83, 0.80 and 0.94, respectively, showing a good discriminatory ability in different cohorts.ConclusionAutoFibroNet is an automated quantitative tool that accurately identifies histological stages of liver fibrosis in Chinese individuals with MAFLD.

Hilar cholangiocarcinoma (HCC) is a common malignant tumor of the biliary system. The structural characteristics of the bile duct tissue can reflect the changes in its function. The visualization of these specific features is of special significance for understanding the degree of invasion of HCC and tumor borders. Radical R0 resection is the only cure for HCC. Currently, the commonly used medical imaging diagnostic methods can only provide a rough tumor range and require a histopathological analysis. Multiphoton microscopy (MPM) not only has an ultra-high spatial resolution but is also extremely sensitive to collagen fibers with noncentrosymmetric structures. In this study, MPM is applied to image the boundaries of HCC tumors. First, the experimental results show that MPM can clearly reveal the existence of residual cancer cells at the surgical margin. Second, the density of the collagen fibers and the dispersion of the 2D direction angle of the collagen fibers of normal and cancerous tissues are further calculated quantitatively. The collagen fiber signals before and after cancer cell invasion into the different tissues are found to be different. Finally, through a logistic regression prediction curve, combined with the collagen fiber density and 2D direction angle as indicators, it is further judged whether the resection margin is negative, which is closely related to the prognosis of HCC. The experimental results indicate that MPM imaging can serve as a new tool for diagnosing whether the HCC tumor boundary reaches R0 resection.

During tumor resection, doctors use intraoperative biopsies to determine the tumor margin. However, the pathological procedures of traditional diagnostic methods, such as imprint cytology and frozen section analysis, are complicated and time-consuming. As this is not conducive to surgeries, their applications are limited to a large extent. Therefore, novel fast microscopy imaging technologies with resolutions comparable to those of pathological tissue sections are necessary. Stimulated Raman scattering (SRS), photoacoustic microscopy (PAM), multiphoton microscopy (MPM), and optical coherence microscopy (OCM) exhibit the advantages of high spatial resolution, large imaging depth, avoiding damage to biological tissues, label-free detection, and the availability of biochemical information of tissues. Additionally, they are superior to intraoperative biopsies owing to their fast imaging speeds. Therefore, they possess broad application prospects in tumor resection surgeries and the diagnosis of other diseases. This study briefly introduces the basic principles, structural characteristics, advantages and disadvantages, and the existing research status of SRS, PAM, MPM, and OCM in biomedicine. Furthermore, we propose a multi-mode hybrid detection technology that can be used for surgeries. The combination of the proposed technology with deep learning-based artificial intelligence can form the basis for intraoperative diagnosis in the future.

Histological grade is one of the most powerful prognostic factors for breast cancer and impacts treatment decisions. However, a label-free and automated classification system for histological grading of breast tumors has not yet been developed. In this study, we employed label-free multiphoton microscopy (MPM) to acquire subcellular-resolution images of unstained breast cancer tissues. Subsequently, a deep-learning algorithm based on the generative adversarial network (GAN) was introduced to learn a representation using only MPM images without the histological grade information. Furthermore, to obtain abundant image information and determine the detailed differences between MPM images of different grades, a multiple-feature discriminator network based on the GAN was leveraged to learn the multi-scale spatial features of MPM images through unlabeled data. The experimental results showed that the classification accuracies for tumors of grades 1, 2, and 3 were 92.4%, 88.6%, and 89.0%, respectively. Our results suggest that the fusion of multiphoton microscopy and the GAN-based deep learning algorithm can be used as a fast and powerful clinical tool for the computer-aided intelligent pathological diagnosis of breast cancer.

No abstract