Classification in the Presence of Label Noise: A Survey

Abstract

Label noise is an important issue in classification, with many potential negative consequences. For example, the accuracy of predictions may decrease, whereas the complexity of inferred models and the number of necessary training samples may increase. Many works in the literature have been devoted to the study of label noise and the development of techniques to deal with label noise. However, the field lacks a comprehensive survey on the different types of label noise, their consequences and the algorithms that consider label noise. This paper proposes to fill this gap. First, the definitions and sources of label noise are considered and a taxonomy of the types of label noise is proposed. Second, the potential consequences of label noise are discussed. Third, label noise-robust, label noise cleansing, and label noise-tolerant algorithms are reviewed. For each category of approaches, a short discussion is proposed to help the practitioner to choose the most suitable technique in its own particular field of application. Eventually, the design of experiments is also discussed, what may interest the researchers who would like to test their own algorithms. In this paper, label noise consists of mislabeled instances: no additional information is assumed to be available like e.g., confidences on labels.

Selecting training sets for support vector machines: a review

“…Other potential issues concerned with the data quality relate to the label and/or feature noise, which can adversely impact the classifier performance. It is especially visible in practical medical applications, in which a majority of diagnostic tests are not 100% accurate, and cannot be considered a gold standard (Frenay and Verleysen 2014) (e.g., there may be discrepancies between the segmentation of the same medical image analyzed by two independent radiology experts). The consequences of the label noise on the behavior of a classifier can be very severe.…”

Data-based fault detection in chemical processes: Managing records with operator intervention and uncertain labels

“…Finally, it is worth to note that various strategies dealing with mislabeling (Frénay & Verleysen, 2014) may be investigated and compared. However, this framework only focuses on the impact of human intervention on FD of a dynamic system, which is a new topic in the chemical engineering literature.…”

“…There are various strategies dealing with mislabeling (Frénay & Verleysen, 2014). The aim of this subsection is the comparison of the proposed approach with a filtering approach.…”

“…The current literature on learning with label noise is a lively mixture of theoretical and experimental studies that clearly demonstrate both the complexity and the importance of the problem. Frénay and Verleysen (2014) comprehensively discussed different families of algorithms that have been proposed to deal with label noise.…”

“…It is worth to mention that most computer science literature dealing with label noise interprets noise as an outlier (Frénay & Verleysen, 2014). In other words, mislabeling is considered for each instance of attributes.…”