Neural Networks-Based On-Site Dermatologic Diagnosis through Hyperspectral Epidermal Images

Abstract

Cancer originates from the uncontrolled growth of healthy cells into a mass. Chromophores, such as hemoglobin and melanin, characterize skin spectral properties, allowing the classification of lesions into different etiologies. Hyperspectral imaging systems gather skin-reflected and transmitted light into several wavelength ranges of the electromagnetic spectrum, enabling potential skin-lesion differentiation through machine learning algorithms. Challenged by data availability and tiny inter and intra-tumoral variability, here we introduce a pipeline based on deep neural networks to diagnose hyperspectral skin cancer images, targeting a handheld device equipped with a low-power graphical processing unit for routine clinical testing. Enhanced by data augmentation, transfer learning, and hyperparameter tuning, the proposed architectures aim to meet and improve the well-known dermatologist-level detection performances concerning both benign-malignant and multiclass classification tasks, being able to diagnose hyperspectral data considering real-time constraints. Experiments show 87% sensitivity and 88% specificity for benign-malignant classification and specificity above 80% for the multiclass scenario. AUC measurements suggest classification performance improvement above 90% with adequate thresholding. Concerning binary segmentation, we measured skin DICE and IOU higher than 90%. We estimated 1.21 s, at most, consuming 5 Watts to segment the epidermal lesions with the U-Net++ architecture, meeting the imposed time limit. Hence, we can diagnose hyperspectral epidermal data assuming real-time constraints.

Acceleration of Hyperspectral Skin Cancer Image Classification through Parallel Machine-Learning Methods

“…The images were collected in two hospitals of the Canary Islands, Spain: the Hospital Universitario de Gran Canaria Doctor Negrín and the Complejo Hospitalario Universitario Insular-Materno Infantil. The image labelling was led by experts such as dermatologists and pathologists according to the taxonomy described in [ 32 ].…”

“…The spectral signatures among different patients have been normalized as illustrated in [ 32 ] to mitigate the variations in illumination conditions. At the end of preprocessing, the spectral signatures contain 116 bands with values in the range [0, 1].…”

“…Several DL models have been adopted in [ 32 ], namely, ResNet-18, ResNet-50, ResNet-101, a ResNet-50 variant, U-Net, and U-Net++ architectures. Since neural networks are time-consuming and computationally expensive, a parallel version of the U-Net++, resulting in the best predictive approach, has been implemented using a low-power NVIDIA Jetson GPU.…”

Intra-operative brain tumor detection with deep learning-optimized hyperspectral imaging

“…69 As a separate part of the HELICoiD project, an in vitro histology dataset was also produced 70 and employed for classification, achieving high-accuracy results with classical and later deep learning methods, 40,71,72 including using superpixel aggregation. 73 The experience gathered by the HELICoiD group also led to the application of similar methods for the classification of skin cancers, 74,75 Alzheimer's disease, 76 gastroenterology, 29 thyroid 42 and ENT cancers. 41,43,77 An independent brain cancer dataset containing 13 images of 12 patients was collected and analyzed by Ref.…”

Hyperspectral imaging acquisition set-up for medical applications

“…We mentioned how optical variables, the motion system, the need for synchronisation, and correct illumination influence the data quality. Another crucial step is the image calibration process 12,14,19 . For each pixel, the hyperspectral camera sensor measures the reflected energy onto the frames constituting the hypercube, each pixel having a digital value.…”