Intra-operative brain tumor detection with deep learning-optimized hyperspectral imaging

Abstract

Surgery for gliomas (intrinsic brain tumors), especially when low-grade, is challenging due to the infiltrative nature of the lesion. Currently, no real-time, intra-operative, label-free and wide-field tool is available to assist and guide the surgeon to find the relevant demarcations for these tumors. While marker-based methods exist for the high-grade glioma case, there is no convenient solution available for the low-grade case; thus, marker-free optical techniques represent an attractive option. Although RGB imaging is a standard tool in surgical microscopes, it does not contain sufficient information for tissue differentiation. We leverage the richer information from hyperspectral imaging (HSI), acquired with a snapscan camera in the 468 − 787 nm range, coupled to a surgical microscope, to build a deep-learning-based diagnostic tool for cancer resection with potential for intra-operative guidance. However, the main limitation of the HSI snapscan camera is the image acquisition time, limiting its widespread deployment in the operation theater. Here, we investigate the effect of HSI channel reduction and pre-selection to scope the design space for the development of cheaper and faster sensors. Neural networks are used to identify the most important spectral channels for tumor tissue differentiation, optimizing the trade-off between the number of channels and precision to enable real-time intra-surgical application. We evaluate the performance of our method on a clinical dataset that was acquired during surgery on five patients. By demonstrating the possibility to efficiently detect low-grade glioma, these results can lead to better cancer resection demarcations, potentially improving treatment effectiveness and patient outcome.

Integrating hyperspectral imaging in an existing intra-operative environment for detection of intrinsic brain tumors

“…This dataset has been used to develop and evaluate the performance of machine learning (ML) algorithms for LGG differentiation. A summary of the data analysis and outcome based on this feasibility data set follows, but the interested reader is referred to the paper by Giannantonio et al [40].…”

“…In the work by Giannantonio et al [40], training and assessment of the classification performance of ML algorithms was done by comparing ground truth annotations of the surgeon to predictions based on the HSI images (see Figure 5). To expand the training data set, and assess the performance on previously unseen data, the hyperspectral image data was split into small tiles grouping hyperspectral voxels sharing similar spectral patterns and intensities.…”

“…The study follows a stop-and-go design, for which recruitment of 8-10 patients is expected during its initial phase. The initial phase of the study is currently on-going and preliminary data from the first five subjects are presented in the work by Giannantonio et al [40].…”

“…Following annotation, the RGB images are spatially co-registered with the HSI data according to the method outlined in Section 2.2. The annotated RGB images act as a ground truth for the HSI data [40].…”

Compact high-speed snapshot multispectral imagers in the VIS/NIR (460 to 960 nm) and SWIR range (1.1 to 1.65 nm) and its potential in a diverse range of applications

“…In more open surgery, Pichette et al [54] used an imec Snapshot VIS camera for real-time monitoring of brain oxygenation during brain surgery. Similarly, after encouraging results with hyperspectral imaging for brain glioma detection and classification [55], [56] the exploration of snapshot spectral cameras for glioma classification is envisaged.…”