Near Real-Time Intraoperative Brain Tumor Diagnosis Using Stimulated Raman Histology and Deep Neural Networks

Abstract

INTRODUCTION Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery. The existing workflow for intraoperative diagnosis based on H&E staining of processed tissue is time-, resource-, and labor-intensive. Moreover, interpretation of intraoperative histologic images is dependent on a pathology workforce that is contracting and unevenly distributed across the centers where cancer surgery is performed worldwide. METHODS We developed an automated workflow, independent of traditional H&E histology, that combines stimulated Raman histology (SRH), a rapid label-free optical imaging method, and deep convolutional neural networks (CNN) to predict diagnosis at the bedside in near real time. Specifically, our CNN, trained on over 2.5 million SRH images, predicts brain tumor diagnosis in the operating room in under 150 s, which is an order of magnitude faster than conventional techniques (eg, 20-30 min). RESULTS To validate our workflow in the clinical setting, we designed a multicenter, prospective, noninferiority clinical trial (N = 204) that compares SRH plus CNN vs traditional H&E histology. Primary endpoint was overall diagnostic accuracy. We show that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% vs 95.5%). Additionally, our CNN learned a hierarchy of interpretable histologic feature representations to classify the major histopathologic classes of brain tumors. We then developed and implemented a semantic segmentation method that can identify tumor infiltrated and diagnostic regions within SRH images. Mean intersection over union values was 61 ± 28.6 for ground truth diagnostic class and 86.0 ± 28.6 for tumor-infiltrated regions. CONCLUSION We have demonstrated how combining bedside optical histology with deep learning can result in near real-time intraoperative brain tumor diagnosis. Our workflow provides a means of delivering expert-level intraoperative diagnosis where neuropathology resources are scarce and improve diagnostic accuracy in resource-rich centers.

Letter: Academic Neurosurgery Department Response to COVID-19 Pandemic: The University of Miami/Jackson Memorial Hospital Model

“…These visits have been replaced by telemedicine clinic visits, which have previously been validated in the neurosurgery population. [5][6][7][8][9] We have implemented Health Insurance Portability and Accountability Act (HIPPA)-compliant telehealth technology integrated into our electronic medical record (Epic, Epic Systems Corporation, Verona, Wisconsin). Neurosurgeons and their patients can communicate via secure video-conference feed using a computer, mobile device, or tablet.…”

Automated histologic diagnosis of CNS tumors with machine learning

“…Most recently, Hollon et al [ 46 ] trained a deep CNN to distinguish among 13 different classes of brain tumors and non-neoplastic brain tissue using intraoperative SRH specimens obtained during brain tumor surgery. The algorithm was able to distinguish malignant glioma, diffuse low-grade gliomas, pilocytic astrocytoma, ependymoma, lymphoma, metastases, medulloblastoma, meningioma, pituitary adenoma, gliosis, white matter, gray matter and nondiagnostic tissue.…”

“…The algorithm was able to distinguish malignant glioma, diffuse low-grade gliomas, pilocytic astrocytoma, ependymoma, lymphoma, metastases, medulloblastoma, meningioma, pituitary adenoma, gliosis, white matter, gray matter and nondiagnostic tissue. This CNN-based automated interpretation of intraoperative SRH images was tested in a multicenter prospective fashion and found to arrive at a diagnosis in less than 150 seconds and with a diagnostic accuracy of 94.6% without human input, which was noninferior to pathologist-based interpretation of conventional intraoperative frozen and smear preparations with an overall accuracy of 93.9% [ 46 ].…”

“…Hollon et al [ 46 ] provided additional insight into the potential role for deep neural networks to aid neuropathologists in improving intraoperative diagnostic accuracy. Of 278 specimens, 17 were misdiagnosed by expert board-certified neuropathologists (93.9% overall accuracy), but the CNN correctly classified all of these 17 difficult specimens.…”

“…The closest achievement of this goal for brain tumors utilized a deep learning approach to interpreting stimulated Raman histology [ 46 ]. SRH has the potential to revolutionize intraoperative histopathology, although the specialized microscope is currently available at a limited number of research institutions.…”