Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis

Abstract

Supervised training of deep learning models requires large labeled datasets. There is a growing interest in obtaining such datasets for medical image analysis applications. However, the impact of label noise has not received sufficient attention. Recent studies have shown that label noise can significantly impact the performance of deep learning models in many machine learning and computer vision applications. This is especially concerning for medical applications, where datasets are typically small, labeling requires domain expertise and suffers from high inter-and intra-observer variability, and erroneous predictions may influence decisions that directly impact human health. In this paper, we first review the state-of-the-art in handling label noise in deep learning. Then, we review studies that have dealt with label noise in deep learning for medical image analysis. Our review shows that recent progress on handling label noise in deep learning has gone largely unnoticed by the medical image analysis community. To help achieve a better understanding of the extent of the problem and its potential remedies, we conducted experiments with three medical imaging datasets with different types of label noise, where we investigated several existing strategies and developed new methods to combat the negative effect of label noise. Based on the results of these experiments and our review of the literature, we have made recommendations on methods that can be used to alleviate the effects of different types of label noise on deep models trained for medical image analysis. We hope that this article helps the medical image analysis researchers and developers in choosing and devising new techniques that effectively handle label noise in deep learning.

Comparing different deep learning architectures for classification of chest radiographs

“…Therefore, it can be assumed, that there is a certain amount of noise in the training data, which might affect the accuracy of the models trained on it. Implementing a human-in-the loop approach for partially correcting the label noise could further improve performance of networks trained on the CheXpert dataset 21 . Our findings differ from applied techniques used in previous literature, where deeper network architectures, mainly a DenseNet-121, were used to classify the CheXpert data set 6,9,22 .…”

Visual-Based Person Detection for Search-and-Rescue with UAS: Humans vs. Machine Learning Algorithm

“…Labelled-image databases are usually used for the training and testing of deep neural networks and were also applied in our research. The research community has recognized the importance of the impact of the label errors (label noise) in training datasets on the model accuracy and have introduced works attempting to understand noisy training labels [25].…”

Ghost Loss to Question the Reliability of Training Data

“…Dealing with label noise in the context of supervised machine learning is a well-known issue that challenges researchers since the early developments of classifiers, as detailed in [1]. A vast corpus of techniques has been developed, most of which are listed in a recent exhaustive survey [7], where even the latest methods related to general deep learning algorithms are indexed. Interestingly, in [7], authors propose to classify techniques for handling label noise into six (possibly overlapping) categories.…”

“…None of the reconstructions based on the feature vector of the ''true class 0'' matches the initial image. Then, the corresponding reconstructions from feature vector ''5'' (5,6,7). When ghost feature vectors are allowed, the network is able to reconstruct the initial image from the feature vector ''5'', which corresponds to a non-true class, while it is not the case when no ghost feature vector is allowed.…”