Deep Learning-Based High-Frequency Ultrasound Skin Image Classification with Multicriteria Model Evaluation

Abstract

This study presents the first application of convolutional neural networks to high-frequency ultrasound skin image classification. This type of imaging opens up new opportunities in dermatology, showing inflammatory diseases such as atopic dermatitis, psoriasis, or skin lesions. We collected a database of 631 images with healthy skin and different skin pathologies to train and assess all stages of the methodology. The proposed framework starts with the segmentation of the epidermal layer using a DeepLab v3+ model with a pre-trained Xception backbone. We employ transfer learning to train the segmentation model for two purposes: to extract the region of interest for classification and to prepare the skin layer map for classification confidence estimation. For classification, we train five models in different input data modes and data augmentation setups. We also introduce a classification confidence level to evaluate the deep model’s reliability. The measure combines our skin layer map with the heatmap produced by the Grad-CAM technique designed to indicate image regions used by the deep model to make a classification decision. Moreover, we propose a multicriteria model evaluation measure to select the optimal model in terms of classification accuracy, confidence, and test dataset size. The experiments described in the paper show that the DenseNet-201 model fed with the extracted region of interest produces the most reliable and accurate results.

High-Frequency Ultrasound Dataset for Deep Learning-Based Image Quality Assessment

“…The first goal of this step is the CNN model selection, providing the most reliable classification results. Based on the previous experiences [4] and the recent papers in medical IQA [38], or informative HFUS image selection [10], we consider two most promising architectures. The first one is DenseNet-201 [50] and the second is VGG16 [51].…”

“…During the last decades, high-frequency ultrasound (HFUS, >20 MHz) has opened up new diagnostic paths in skin analysis, enabling visualization and diagnosis of superficial structures [1,2]. Therefore, it has gained popularity in various areas of medical diagnostics [3,4] and is now commonly used in medical practice [5]. In oncology, it helps in the determination of skin tumor depth, prognosis, and surgical planning [1,6], enabling differentiation between melanoma, benign nevi, and seborrheic keratoses [7].…”

“…Usually, the development of imaging techniques is followed by the fast development of dedicated image processing algorithms. In recent years, there appeared in literature [2,4,10,[13][14][15] different solutions in computer-aided diagnosis (CAD) of skin in HFUS data, which target segmentation, detection, and classification of the affected areas. A robust skin layer segmentation in HFUS images was as first described by Gao et al [16], and developed by Sciolla et al [15], to finally gain Dice index of 0.919 in [17] for epidermis segmentation, and 0.934 for fetus body segmentation in embryonic mice HFUS volume image analysis.…”

“…An extension to this targeting skin layer segmentation for accurate tumor seed point detection is described in [14]. The first solution of HFUS image classification is described in [4], where the considered cases include inflammatory skin diseases, skin tumors, and control group. All the mentioned techniques assume that the preselected correctly acquired images are proceeded and do not evaluate the incorrect input data.…”

“…Since the development of HFUS image analysis coincided with the dynamic development of machine learning algorithms, especially in the area of deep learning, most of the newest approaches [4,10,17] utilize their benefits. The first applications of convolutional neural network (CNN) to HFUS image segmentation are described in [2,19].…”