Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning

Halicek, Martin; Dormer, James D.; Little, James V.; Chen, Amy Y.; Fei, Baowei

doi:10.1364/boe.381257

Cited by 67 publications

(62 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to provide a comparison of performance between HSI and RGB imagery, we performed the classification of synthetic RGB images using the same CNN. Such RGB images were extracted from the HS data, where each color channel was generated equalizing the spectral information to match the spectral response of the human eye [30]. After separately training the CNN with RGB patches, the models selected after the validation were found to be competitive.…”

Section: Validation Resultsmentioning

confidence: 99%

Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

Ortega

Halicek

Fabelo

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

Hyperspectral imaging (HSI) technology has demonstrated potential to provide useful information about the chemical composition of tissue and its morphological features in a single image modality. Deep learning (DL) techniques have demonstrated the ability of automatic feature extraction from data for a successful classification. In this study, we exploit HSI and DL for the automatic differentiation of glioblastoma (GB) and non-tumor tissue on hematoxylin and eosin (H&E) stained histological slides of human brain tissue. GB detection is a challenging application, showing high heterogeneity in the cellular morphology across different patients. We employed an HSI microscope, with a spectral range from 400 to 1000 nm, to collect 517 HS cubes from 13 GB patients using 20× magnification. Using a convolutional neural network (CNN), we were able to automatically detect GB within the pathological slides, achieving average sensitivity and specificity values of 88% and 77%, respectively, representing an improvement of 7% and 8% respectively, as compared to the results obtained using RGB (red, green, and blue) images. This study demonstrates that the combination of hyperspectral microscopic imaging and deep learning is a promising tool for future computational pathologies.

show abstract

Section: Validation Resultsmentioning

confidence: 99%

Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

Ortega

Halicek

Fabelo

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multispectral tissue differentiation has become intensively studied and analyzed using machine learning methods as these can catch high-dimensional tissue behavior [ 55 , 56 , 57 ]. In this study, knowledge about the spectral behavior of cholesteatoma and bone, as well as only a few spectral bands are used.…”

Section: Discussionmentioning

confidence: 99%

Surgical Guidance for Removal of Cholesteatoma Using a Multispectral 3D-Endoscope

Wisotzky

Rosenthal

Wege

et al. 2020

Sensors

View full text Add to dashboard Cite

We develop a stereo-multispectral endoscopic prototype in which a filter-wheel is used for surgical guidance to remove cholesteatoma tissue in the middle ear. Cholesteatoma is a destructive proliferating tissue. The only treatment for this disease is surgery. Removal is a very demanding task, even for experienced surgeons. It is very difficult to distinguish between bone and cholesteatoma. In addition, it can even reoccur if not all tissue particles of the cholesteatoma are removed, which leads to undesirable follow-up operations. Therefore, we propose an image-based method that combines multispectral tissue classification and 3D reconstruction to identify all parts of the removed tissue and determine their metric dimensions intraoperatively. The designed multispectral filter-wheel 3D-endoscope prototype can switch between narrow-band spectral and broad-band white illumination, which is technically evaluated in terms of optical system properties. Further, it is tested and evaluated on three patients. The wavelengths 400 nm and 420 nm are identified as most suitable for the differentiation task. The stereoscopic image acquisition allows accurate 3D surface reconstruction of the enhanced image information. The first results are promising, as the cholesteatoma can be easily highlighted, correctly identified, and visualized as a true-to-scale 3D model showing the patient-specific anatomy.

show abstract

“…In HSI digital histology, Ortega et al detected glioblastoma brain cancer in digital slides using a patch-based 2D-CNN approach [70]. Additionally, Halicek et al has employed very deep 2D-CNNs for classification, specifically the widely-used Inception v4 model (Figure 4) implemented in a sliding patch-based approach for head and neck squamous cancer [71] and thyroid and salivary gland cancers [72]. For comparing 2D-CNN and 3D-CNNs, in [73] Halicek et al explored spatial-spectral convolutions in 3D CNNs with 3D convolutional kernels to 2D approaches.…”

Section: Deep Learning Methodsmentioning

confidence: 99%

“…In-vivo brain tumor detection ‡ [27] Deep learning 2D-CNN and 1D-DNN In-vivo brain tumor detection ‡ [69] 2D-CNN (Inception v4) Head and neck cancer [71] Salivary gland cancer [72] 2D-CNN and 3D-CNN Head and neck cancer [73] 2D-CNN (U-Net) Tongue cancer detection [74] Breast cancer [75] GAN HS image generation from RGB [76] RNNs, 2D-CNN and 3D-CNN Head and neck cancer detection [77] Publicly available datasets are marked with ‡ .…”

Section: Spatial and Spectral Features In Supervised Classificationmentioning

confidence: 99%

Information Extraction Techniques in Hyperspectral Imaging Biomedical Applications

Ortega¹,

Halicek²,

Fabelo³

et al. 2021

Multimedia Information Retrieval

Self Cite

View full text Add to dashboard Cite

Hyperspectral imaging (HSI) is a technology able to measure information about the spectral reflectance or transmission of light from the surface. The spectral data, usually within the ultraviolet and infrared regions of the electromagnetic spectrum, provide information about the interaction between light and different materials within the image. This fact enables the identification of different materials based on such spectral information. In recent years, this technology is being actively explored for clinical applications. One of the most relevant challenges in medical HSI is the information extraction, where image processing methods are used to extract useful information for disease detection and diagnosis. In this chapter, we provide an overview of the information extraction techniques for HSI. First, we introduce the background of HSI, and the main motivations of its usage for medical applications. Second, we present information extraction techniques based on both light propagation models within tissue and machine learning approaches. Then, we survey the usage of such information extraction techniques in HSI biomedical research applications. Finally, we discuss the main advantages and disadvantages of the most commonly used image processing approaches and the current challenges in HSI information extraction techniques in clinical applications.

show abstract

Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning

Cited by 67 publications

References 37 publications

Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

Surgical Guidance for Removal of Cholesteatoma Using a Multispectral 3D-Endoscope

Information Extraction Techniques in Hyperspectral Imaging Biomedical Applications

Contact Info

Product

Resources

About