Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection

Abstract

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging

“…To compute statistically significant differences, a two-tailed paired t-test (p = 0.05) similar to previous works in HSI classification analysis using Excel (Microsoft Office 365, Microsoft, Redmond, WA, USA), was used 37 .…”

Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology

“…Based on a previous animal study from Barberio et al comparing the two most successful deep learning models, i.e., support vector machines (SVMs) [ 52 , 53 , 54 ] and CNNs for HSI tissue segmentation, the CNN model achieved an overall higher sensitivity for all tissue classes (89.4%) except for the nerve class, which had a sensitivity of 76.3% [ 46 ]. Consequently, this combined with deep learning allowed for the automatic discrimination of six different tissue classes (artery, vein, adipose, muscle, skin, and nerve), supporting our decision to adopt the CNN model in this study.…”

“…Using a four-layer neural network, Boris Jansen-Winkeln et al achieved an 86% sensitivity and a 95% specificity in identifying colorectal cancer (CRC) [ 45 ]. Barberio et al showed that HSI combined with CNNs could be used to automatically recognize key anatomical structures such as blood vessels and nerves [ 46 ]. Collins and Maktabi [ 47 ] showed that colorectal and esophagogastric cancer could also be detected with various machine learning models, and CNNs often produced the best results.…”

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

“…However, the accuracy of the automatic detection was rather low to be acceptable within a clinical setting. For that reason, another study was performed [34] focusing on automatic tissue recognition in animals. During neck dissections in several pigs, in vivo HSI acquisitions were made, and different tissue classes (such as muscle, nerve, vessels, skin, etc.)…”