Multi-task Curriculum Framework for Open-Set Semi-supervised Learning

Yu, Qing; Ikami, Daiki; Irie, Go; Aizawa, Kiyoharu

doi:10.1007/978-3-030-58610-2_26

Cited by 64 publications

(79 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their superior results also imply the efficacy of SSL methods on UDA tasks. We note that some works [33,52] presented that SSL methods performed poorly when there was a class distribution shift across labeled and unlabeled data; in contrast, we find that SSL methods perform well when there is a marginal distribution shift (i.e., under the covariate shift assumption 1), facilitating a deeper understanding of their application fields.…”

Section: Strong Uda Baselines With Ssl Methodsmentioning

confidence: 53%

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

Zhang,

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

Unsupervised domain adaptation (UDA) and semi-supervised learning (SSL) are two typical strategies to reduce expensive manual annotations in machine learning. In order to learn effective models for a target task, UDA utilizes the available labeled source data, which may have different distributions from unlabeled samples in the target domain, while SSL employs few manually annotated target samples. Although UDA and SSL are seemingly very different strategies, we find that they are closely related in terms of task objectives and solutions, and SSL is a special case of UDA problems. Based on this finding, we further investigate whether SSL methods work on UDA tasks. By adapting eight representative SSL algorithms on UDA benchmarks, we show that SSL methods are strong UDA learners. Especially, state-of-the-art SSL methods significantly outperform existing UDA methods on the challenging UDA benchmark of DomainNet, and state-of-the-art UDA methods could be further enhanced with SSL techniques. We thus promote that SSL methods should be employed as baselines in future UDA studies and expect that the revealed relationship between UDA and SSL could shed light on future UDA development.Codes are available at https://github.com/YBZh.Preprint. Under review.

show abstract

Section: Strong Uda Baselines With Ssl Methodsmentioning

confidence: 53%

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

Zhang,

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The combination of curriculum learning and semi-supervised learning is popular in recent years [37][38][39]. For multi-model image classification task, [37] optimized the learning process of unlabeled images by judging their reliability and discriminability.…”

Section: Related Workmentioning

confidence: 99%

“…In [38], the easy image-level properties are learned first and then used to facilitate segmentation via constrained CNNs. Curriculum learning is also used to alleviate out-of-distribution problems by picking up in-distribution samples from unlabeled data according to the out-of-distribution scores [39].…”

Section: Related Workmentioning

confidence: 99%

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

Zhang

Wang

Hou

et al. 2021

Preprint

View full text Add to dashboard Cite

The recently proposed FixMatch achieved state-of-the-art results on most semisupervised learning (SSL) benchmarks. However, like other modern SSL algorithms, FixMatch uses a pre-defined constant threshold for all classes to select unlabeled data that contribute to the training, thus failing to consider different learning status and learning difficulties of different classes. To address this issue, we propose Curriculum Pseudo Labeling (CPL), a curriculum learning approach to leverage unlabeled data according to the model's learning status. The core of CPL is to flexibly adjust thresholds for different classes at each time step to let pass informative unlabeled data and their pseudo labels. CPL does not introduce additional parameters or computations (forward or backward propagation). We apply CPL to FixMatch and call our improved algorithm FlexMatch. Flex-Match achieves state-of-the-art performance on a variety of SSL benchmarks, with especially strong performances when the labeled data are extremely limited or when the task is challenging. For example, FlexMatch outperforms FixMatch by 14.32% and 24.55% on CIFAR-100 and STL-10 datasets respectively, when there are only 4 labels per class. CPL also significantly boosts the convergence speed, e.g., FlexMatch can use only 1/5 training time of FixMatch to achieve even better performance. Furthermore, we show that CPL can be easily adapted to other SSL algorithms and remarkably improve their performances. We open source our code at https://github.com/TorchSSL/TorchSSL.

show abstract

“…Ideally, a model should classify samples of known categories i.e., inliers, into correct classes while identifying samples of novel categories as outliers. This task is called Open-set Semi-supervised Learning (OSSL) [44]. While OSSL is a more realistic and practical scenario than standard SSL, it has not been as widely explored.…”

Section: Introductionmentioning

confidence: 99%

OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers

Saito,

Kim,

Saenko

2021

Preprint

View full text Add to dashboard Cite

Semi-supervised learning (SSL) is an effective means to leverage unlabeled data to improve a model's performance. Typical SSL methods like FixMatch assume that labeled and unlabeled data share the same label space. However, in practice, unlabeled data can contain categories unseen in the labeled set, i.e., outliers, which can significantly harm the performance of SSL algorithms. To address this problem, we propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch. Learning representations of inliers while rejecting outliers is essential for the success of OSSL. To this end, OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers. The OVA-classifier outputs the confidence score of a sample being an inlier, providing a threshold to detect outliers. Another key contribution is an open-set soft-consistency regularization loss, which enhances the smoothness of the OVA-classifier with respect to input transformations and greatly improves outlier detection. OpenMatch achieves stateof-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.Preprint. Under review.

show abstract

Multi-task Curriculum Framework for Open-Set Semi-supervised Learning

Cited by 64 publications

References 10 publications

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers

Contact Info

Product

Resources

About