Multi-view adaptive semi-supervised feature selection with the self-paced learning

Shi, Caijuan; Gu, Zhibin; Duan, Changyu; Tian, Qi

doi:10.1016/j.sigpro.2019.107332

“…Self-paced learning [42], [43] can filter out noisy labels by assigning small weights to mislabeled samples and large weights to clean samples, thus ensuring robust model learning. Specifically, specifying a monotonically decreasing weighting function allows the classifier to focus on the easy samples first and then fit the difficult samples.…”

Section: Sample Selection Approachmentioning

confidence: 99%

Training Robust Deep Neural Networks on Noisy Labels Using Adaptive Sample Selection With Disagreement

Takeda

¹

,

Yoshida

²

,

Muneyasu

³

2021

IEEE Access

3

0

View full text Add to dashboard Cite

Learning with noisy labels is one of the most practical but challenging tasks in deep learning. One promising way to treat noisy labels is to use the small-loss trick based on the memorization effect, that is, clean and noisy samples are identified by observing the network's loss during training. Co-teaching+ is a state-of-the-art method that simultaneously trains two networks with small-loss selection using the "update by disagreement" strategy; however, it suffers from the problem that the selected samples tend to become noisy as the number of iterations increases. This phenomenon means that clean small-loss samples will be biased toward agreement data, which is the set of samples for which the two networks have the same prediction. This paper proposes an adaptive sample selection method to train deep neural networks robustly and prevent noise contamination in the disagreement strategy. Specifically, the proposed method calculates the threshold of the small-loss criterion by considering the loss distribution of the whole batch at each iteration. Then, the network is backpropagated by extracting samples below this threshold from the disagreement data. Combining the disagreement and agreement data of the two networks can suppress the degradation of the true-label rate of training data in a mini batch. Experiments were conducted using five commonly used benchmarks, MNIST, CIFAR-10, CIFAR-100, NEWS, and T-ImageNet to verify the robustness of the proposed method to noisy labels. The results show the proposed method improves generalization performance in an image classification task with simulated noise rates of up to 50%.INDEX TERMS Deep neural network, learning with noisy labels, image classification, co-teaching.

show abstract

“…The semi-supervised feature selection method uses data with labels and unlabeled; in contrast, the other choice of semi-supervised method is the pairwise constraint. In this method, not all data sets have labels, but there is side information like a pairwise constraint [12,13]. A pairwise constraint is a pair of data belonging to the different clusters (cannot-link) or the same cluster (must-link) [14].…”

Section: Introductionmentioning

confidence: 99%

A Novel Method of Constrained Feature Selection by the Measurement of Pairwise Constraints Uncertainty

Berahmand

¹

,

Rostami

²

,

Forouzandeh

³

2020

Preprint

1

0

View full text Add to dashboard Cite

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. Semi-supervised learning is a class of machine learning in which unlabeled data and labeled data are used simultaneously to improve feature selection. The goal of feature selection over partially labeled data (semi-supervised feature selection) is to choose a subset of available features with the lowest redundancy with each other and the highest relevancy to the target class, which is the same objective as the feature selection over entirely labeled data. This method actually used the classification to reduce ambiguity in the range of values. First, the similarity values of each pair are collected, and then these values are divided into intervals, and the average of each interval is determined. In the next step, for each interval, the number of pairs in this range is counted. Finally, by using the strength and similarity matrices, a new constraint feature selection ranking is proposed. The performance of the presented method was compared to the performance of the state-of-the-art, and well-known semi-supervised feature selection approaches on eight datasets. The results indicate that the proposed approach improves previous related approaches with respect to the accuracy of the constrained score. In particular, the numerical results showed that the presented approach improved the classification accuracy by about 3% and reduced the number of selected features by 1%. Consequently, it can be said that the proposed method has reduced the computational complexity of the machine learning algorithm despite increasing the classification accuracy.

show abstract

“…The semi-supervised feature selection method uses data with labels and unlabeled; in contrast, the other choice of semi-supervised method is the pairwise constraint. In this method, not all data sets have labels, but there is side information like a pairwise constraint [12,13].…”

mentioning

confidence: 99%

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

Rostami

¹

,

Berahmand

²

,

Forouzandeh

³

2020

View full text Add to dashboard Cite

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. Semi-supervised learning is a class of machine learning in which unlabeled data and labeled data are used simultaneously to improve feature selection. The goal of feature selection over partially labeled data (semi-supervised feature selection) is to choose a subset of available features with the lowest redundancy with each other and the highest relevancy to the target class, which is the same objective as the feature selection over entirely labeled data. This method actually used the classification to reduce ambiguity in the range of values. First, the similarity values of each pair are collected, and then these values are divided into intervals, and the average of each interval is determined. In the next step, for each interval, the number of pairs in this range is counted. Finally, by using the strength and similarity matrices, a new constraint feature selection ranking is proposed. The performance of the presented method was compared to the performance of the state-of-the-art, and well-known semi-supervised feature selection approaches on eight datasets. The results indicate that the proposed approach improves previous related approaches with respect to the accuracy of the constrained score. In particular, the numerical results showed that the presented approach improved the classification accuracy by about 3% and reduced the number of selected features by 1%. Consequently, it can be said that the proposed method has reduced the computational complexity of the machine learning algorithm despite increasing the classification accuracy.

show abstract

Multi-view adaptive semi-supervised feature selection with the self-paced learning

Cited by 29 publications

References 17 publications

Training Robust Deep Neural Networks on Noisy Labels Using Adaptive Sample Selection With Disagreement

Training Robust Deep Neural Networks on Noisy Labels Using Adaptive Sample Selection With Disagreement

A Novel Method of Constrained Feature Selection by the Measurement of Pairwise Constraints Uncertainty

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

Contact Info

Product

Resources

About