Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning

Abstract

In the case of hepatocellular carcinoma (HCC) samples, classification of differentiation is crucial for determining prognosis and treatment strategy decisions. However, a label‐free and automated classification system for HCC grading has not been yet developed. Hence, in this study, we demonstrate the fusion of multiphoton microscopy and a deep‐learning algorithm for classifying HCC differentiation to produce an innovative computer‐aided diagnostic method. Convolutional neural networks based on the VGG‐16 framework were trained using 217 combined two‐photon excitation fluorescence and second‐harmonic generation images; the resulting classification accuracy of the HCC differentiation grade was over 90%. Our results suggest that a combination of multiphoton microscopy and deep learning can realize label‐free, automated methods for various tissues, diseases and other related classification problems.

Deep learning a boon for biophotonics?

“…This technique focuses on transferring features of a deep neural network learned on a larger dataset to a small dataset. Research has shown that transfer‐learning strategies lead to promising results when applied for small spectroscopic dataset . However, transferring features of a deep neural network which is pretrained on a dataset like ImageNet, to perform classification or regression tasks on spectroscopic data, is debatable.…”

“…For cell localization task, a leukocyte mask was generated using a CNN to localize and segment leukocytes in blood smear images (lower panel). These images are reproduced and modified from references . CNN, convolution neural network…”

“…Therefore, the training of MLPs or CNNs is always challenging due to the small datasets available. In such cases, deep learning networks using transfer learning strategies can be applied . Transfer learning utilizes CNN models pretrained on a (large) source dataset and transfer the learned features to classify a (small) target dataset.…”

“…Transfer learning utilizes CNN models pretrained on a (large) source dataset and transfer the learned features to classify a (small) target dataset. For example, pretrained CNN models including GoogleNet , Inceptionv3 and VGG16 were used to classify breast cancer in OCT images , head and neck cancer in 3‐D OCT images , lung cancer in CARS images and hepatocellular carcinoma in multiphoton microscopic images (see Figure top), respectively. Here, the CNN models are first trained on large nonbiological datasets like ImageNet and the parameters of these pretrained models are fine‐tuned on the new biophotonic dataset.…”

“…To summarize, MLPs and CNNs are majorly used for disease classification. Recently reported transfer learning strategies using CNNs have worked best for small datasets, which are accounted in biophotonic studies.…”