Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

Xia, Xiaobo; Liu, Tongliang; Han, Bo; Gong, Mingming; Yu, Jun; Niu, Gang; Sugiyama, Masashi

doi:10.48550/arxiv.2106.00445

Cited by 4 publications

(11 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many historical parameters are used in the loss calculation and update process, therefore loss itself could be regarded as a historical collection. Some studies have used the unique properties of the loss to identify anomalous samples [139] or biased samples [140]- [145]. For instance, Chu et al [139] use loss profiles as an informative cue for detecting anomalies, while [140] and [141] record the loss curve of each sample as an additional attribute to help identify the biased samples.…”

Section: Publication Methodsmentioning

confidence: 99%

“…Inside, cyclical training records the historical loss of each sample, then calculates and ranks every sample's normalized average loss to recognize the noisy data with a higher score. Later, some works introduce the value of historical loss to determine whether the data is with noisy label [142]- [145]. To achieve a better performance of cleaning samples, Xu et al [142] also use the historical loss distribution to handle the issue of varying scales of loss distributions across different training periods.…”

Section: Publication Methodsmentioning

confidence: 99%

“…Although average loss has shown promising results [141], the existence of outliers in the training data can result in deviations when cleaning the data. Hence Xia et al [145] construct two more robust mean estimators to reduce the uncertainty of small-loss and large-loss examples separately. Furthermore, Yue et al [143] explore the algorithm for cleaning the labels based on historical loss, efficiently using historical information to obtain high-quality and noise-free data.…”

Section: Publication Methodsmentioning

confidence: 99%

“…For the third aspect, some studies employ the unique properties of the loss to clean the dataset. They [139]- [145] leverage the loss curve to reweight the loss function so as to identify anomalous/biased samples, and ultimately improve data cleanliness. Last, the fourth aspect focuses on employing historical input to perform data augmentation.…”

Section: Aspect Of Functional Partmentioning

confidence: 99%

“…(5) Loss values. The statistics of loss values are used to identify anomalous data [139] and biased data [140]- [145]. In addition to these historical types, we also consider the functional part (where the data is used) and the storage form (how it is stored) of historical statistics to better understand how it is applied in these methods.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Survey of Historical Learning: Learning Models with Learning History

Li¹,

Ge²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

New knowledge originates from the old. The various types of elements, deposited in the training history, are a large amount of wealth for improving learning deep models. In this survey, we comprehensively review and summarize the topic-"Historical Learning: Learning Models with Learning History", which learns better neural models with the help of their learning history during its optimization, from three detailed aspects: Historical Type (what), Functional Part (where) and Storage Form (how). To our best knowledge, it is the first survey that systematically studies the methodologies which make use of various historical statistics when training deep neural networks. The discussions with related topics like recurrent/memory networks, ensemble learning, and reinforcement learning are demonstrated. We also expose future challenges of this topic and encourage the community to pay attention to the think of historical learning principles when designing algorithms. The paper list related to historical learning is available at https://github.com/Martinser/Awesome-Historical-Learning.

show abstract

Section: Publication Methodsmentioning

confidence: 99%

Section: Publication Methodsmentioning

confidence: 99%

Section: Publication Methodsmentioning

confidence: 99%

Section: Aspect Of Functional Partmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Survey of Historical Learning: Learning Models with Learning History

Li¹,

Ge²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Separating hard clean samples from noisy samples with samples’ learning risk for DNN when learning with noisy labels

Deng,

Yang,

Kang

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

Learning with Noisy Labels (LNL) methods aim to improve the accuracy of Deep Neural Networks (DNNs) when the training set contains samples with noisy or incorrect labels, and have become popular in recent years. Existing popular LNL methods frequently regard samples with high learning difficulty (high-loss and low prediction probability) as noisy samples; however, irregular feature patterns from hard clean samples can also cause high learning difficulty, which can lead to the misclassification of hard clean samples as noisy samples. To address this insufficiency, we propose the Samples’ Learning Risk-based Learning with Noisy Labels (SLRLNL) method. Specifically, we propose to separate noisy samples from hard clean samples using samples’ learning risk, which represents samples’ influence on DNN’s accuracy . We show that samples’ learning risk is comprehensively determined by samples’ learning difficulty as well as samples’ feature similarity to other samples, and thus, compared to existing LNL methods that solely rely on the learning difficulty, our method can better separate hard clean samples from noisy samples, since the former frequently possess irregular feature patterns. Moreover, to extract more useful information from samples with irregular feature patterns (i.e., hard samples), we further propose the Relabeling-based Label Augmentation (RLA) process to prevent the memorization of hard noisy samples and better learn the hard clean samples, thus enhancing the learning for hard samples. Empirical studies show that samples’ learning risk can identify noisy samples more accurately, and the RLA process can enhance the learning for hard samples. To evaluate the effectiveness of our method, we compare it with popular existing LNL methods on CIFAR-10, CIFAR-100, Animal-10N, Clothing1M, and Docred. The experimental results indicate that our method outperforms other existing methods. The source code for SLRLNL can be found at https://github.com/yangbo1973/SLRLNL.

show abstract

Regroup Median Loss for Combating Label Noise

Li,

Tian

et al. 2024

AAAI

View full text Add to dashboard Cite

The deep model training procedure requires large-scale datasets of annotated data. Due to the difficulty of annotating a large number of samples, label noise caused by incorrect annotations is inevitable, resulting in low model performance and poor model generalization. To combat label noise, current methods usually select clean samples based on the small-loss criterion and use these samples for training. Due to some noisy samples similar to clean ones, these small-loss criterion-based methods are still affected by label noise. To address this issue, in this work, we propose Regroup Median Loss (RML) to reduce the probability of selecting noisy samples and correct losses of noisy samples. RML randomly selects samples with the same label as the training samples based on a new loss processing method. Then, we combine the stable mean loss and the robust median loss through a proposed regrouping strategy to obtain robust loss estimation for noisy samples. To further improve the model performance against label noise, we propose a new sample selection strategy and build a semi-supervised method based on RML. Compared to state-of-the-art methods, for both the traditionally trained and semi-supervised models, RML achieves a significant improvement on synthetic and complex real-world datasets. The source is at https://github.com/Feng-peng-Li/Regroup-Loss-Median-to-Combat-Label-Noise.

show abstract

Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

Cited by 4 publications

References 25 publications

A Survey of Historical Learning: Learning Models with Learning History

A Survey of Historical Learning: Learning Models with Learning History

Separating hard clean samples from noisy samples with samples’ learning risk for DNN when learning with noisy labels

Regroup Median Loss for Combating Label Noise

Contact Info

Product

Resources

About