Automated classification of pancreatic neuroendocrine tumors using label-free multiphoton microscopy and deep learning

Abstract

Pancreatic neuroendocrine tumors (PNETs) present significant diagnostic and therapeutic challenges due to their heterogeneity and complex nature as a subtype of pancreatic cancer. The treatment approach varies considerably based on the tumor's location, grading, and focality. Accurate prognosis and management typically necessitate the expertise of a pathologist to evaluate histological slides of the tissue, a process that is often time-consuming and labor-intensive. Developing point-of-care techniques for automatic classification of PNETs would greatly improve the ability to treat and manage this disease by providing real-time decision-making information. In response to these challenges, our study introduces a highly efficient and versatile diagnostic strategy. This innovative approach synergistically integrates labelfree multiphoton microscopy with finely adjusted, pre-trained deep learning models, optimized for performance even with limited data availability. We have meticulously optimized four pre-trained convolutional neural networks, utilizing a dataset comprising only 49 images, which includes both two-photon excitation fluorescence and second-harmonic generation imaging. This refined approach has resulted in an impressive average classification accuracy of over 95% for the development dataset and more than 90% for the test dataset. These results are significantly superior when compared to the preoperative misdiagnosis rates of conventional diagnostic modalities such as ultrasound (US) and computed tomography (CT), which stand at 81.8% and 61.5%, respectively. This methodology represents a significant advancement in the diagnostic process for PNETs, promising a more streamlined, rapid, and accurate approach to treatment. Furthermore, it opens substantial potential for the automated classification of various tumor types using multiphoton microscopic imaging, even in scenarios characterized by limited data availability.