Skin Lesion Classification Using CNNs With Patch-Based Attention and Diagnosis-Guided Loss Weighting

Abstract

Objective: This work addresses two key problems of skin lesion classification. The first problem is the effective use of high-resolution images with pretrained standard architectures for image classification. The second problem is the high class imbalance encountered in real-world multi-class datasets. Methods: To use high-resolution images, we propose a novel patch-based attention architecture that provides global context between small, high-resolution patches. We modify three pretrained architectures and study the performance of patch-based attention. To counter class imbalance problems, we compare oversampling, balanced batch sampling, and class-specific loss weighting. Additionally, we propose a novel diagnosis-guided loss weighting method which takes the method used for groundtruth annotation into account. Results: Our patch-based attention mechanism outperforms previous methods and improves the mean sensitivity by 7 %. Class balancing significantly improves the mean sensitivity and we show that our diagnosis-guided loss weighting method improves the mean sensitivity by 3 % over normal loss balancing. Conclusion: The novel patch-based attention mechanism can be integrated into pretrained architectures and provides global context between local patches while outperforming other patch-based methods. Hence, pretrained architectures can be readily used with high-resolution images without downsampling. The new diagnosis-guided loss weighting method outperforms other methods and allows for effective training when facing class imbalance. Significance: The proposed methods improve automatic skin lesion classification. They can be extended to other clinical applications where high-resolution image data and class imbalance are relevant.

Effective Melanoma Recognition Using Deep Convolutional Neural Network with Covariance Discriminant Loss

“…However, the ground truths of image segmentation are difficult to acquire, limiting the application of these methods. Furthermore, the data imbalance has been addressed in References [ 15 , 16 ]. In Reference [ 15 ], a linear classifier RankOpt was used to tackle the imbalanced data distribution.…”

“…In Reference [ 15 ], a linear classifier RankOpt was used to tackle the imbalanced data distribution. In Reference [ 16 ], a weighted loss, namely the diagnosis-guided loss, was employed to strengthen the classification ability on melanoma.…”

“…Nonetheless, the manual inspection for dermoscopy images is subjective and experimental. Numerous computer-aided diagnosis methods have been presented to perform melanoma recognition [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ].…”

“…Recently, deep convolutional neural networks (DCNNs) have demonstrated the excellent feature representation ability in the computer vision field. Various methods [ 2 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ] adopted DCNNs for melanoma recognition. Owing to the strong representation ability, the DCNN-based methods can generally perform the classification task effectively.…”

“…Owing to the strong representation ability, the DCNN-based methods can generally perform the classification task effectively. In References [ 15 , 16 ], data imbalance is addressed with the cost-sensitive loss and Area Under the receiver operating characteristics Curve (AUC) statistics, which is directly derived from the statistic information of the input dataset. In References [ 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ], data-level methods, algorithm-level methods, and hybrid methods were employed to address the data imbalance, boosting the performance to some extent.…”