RECORD: Resource Constrained Semi-Supervised Learning under Distribution Shift

Guo, Lan-Zhe; Zhou, Zhi; Li, Yufeng

doi:10.1145/3394486.3403214

Cited by 14 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Sconf-Unbiased, Sconf-ABS and Sconf-NN are short for ERM with risk estimators in Eqs. ( 6), (8), and (9), respectively.…”

Section: Methodsmentioning

confidence: 99%

“…However, the collection of massive data with exact supervision is laborious and expensive in many real-world problems. To overcome this bottleneck, weakly supervised learning [1] has been proposed and explored under various settings, including but not limited to, semi-supervised learning [2,3,4,5,6,7,8], positive-unlabeled learning [9,10,11,12], noisy-label learning [13,14,15,16], partial-label learning [17,18,19,20], complementary-label learning [21,22,23,24,25,26], similarity-unlabeled learning [27], and similarity-dissimilarity learning [28].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning from Similarity-Confidence Data

Cao,

Feng,

et al. 2021

Preprint

View full text Add to dashboard Cite

Weakly supervised learning has drawn considerable attention recently to reduce the expensive time and labor consumption of labeling massive data. In this paper, we investigate a novel weakly supervised learning problem of learning from similarityconfidence (Sconf) data, where we aim to learn an effective binary classifier from only unlabeled data pairs equipped with confidence that illustrates their degree of similarity (two examples are similar if they belong to the same class). To solve this problem, we propose an unbiased estimator of the classification risk that can be calculated from only Sconf data and show that the estimation error bound achieves the optimal convergence rate. To alleviate potential overfitting when flexible models are used, we further employ a risk correction scheme on the proposed risk estimator. Experimental results demonstrate the effectiveness of the proposed methods.

show abstract

“…Sconf-Unbiased, Sconf-ABS and Sconf-NN are short for ERM with risk estimators in Eqs. ( 6), (8), and (9), respectively.…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning from Similarity-Confidence Data

Cao,

Feng,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…There are some SSL studies trying to tackle sub-problems in dynamic environments, such as classifying the new emerging labels [144], [145], adapting to gradually shifting distributions [146], [147]. But these attempts mainly focused on specific small problems and are still a long way from being applied to real-world tasks with dynamic environments.…”

Section: Learning In Dynamic Environmentsmentioning

confidence: 99%

Robust Deep Semi-Supervised Learning: A Brief Introduction

Guo¹,

Zhou²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Semi-supervised learning (SSL) is the branch of machine learning that aims to improve learning performance by leveraging unlabeled data when labels are insufficient. Recently, SSL with deep models has proven to be successful on standard benchmark tasks. However, they are still vulnerable to various robustness threats in real-world applications as these benchmarks provide perfect unlabeled data, while in realistic scenarios, unlabeled data could be corrupted. Many researchers have pointed out that after exploiting corrupted unlabeled data, SSL suffers severe performance degradation problems. Thus, there is an urgent need to develop SSL algorithms that could work robustly with corrupted unlabeled data. To fully understand robust SSL, we conduct a survey study. We first clarify a formal definition of robust SSL from the perspective of machine learning. Then, we classify the robustness threats into three categories: i) distribution corruption, i.e., unlabeled data distribution is mismatched with labeled data; ii) feature corruption, i.e., the features of unlabeled examples are adversarially attacked; and iii) label corruption, i.e., the label distribution of unlabeled data is imbalanced. Under this unified taxonomy, we provide a thorough review and discussion of recent works that focus on these issues. Finally, we propose possible promising directions within robust SSL to provide insights for future research.

show abstract

“…In supervised learning, the annotations of a huge number of instances may not be easily obtained in many practical applications due to the concerns including but not limited to time consumption, expenditure, and privacy preserving. For these reasons, many weakly supervised learning (WSL) frameworks [1] have been studied in various scenarios recently, including semi-supervised learning [2,3,4,5,6,7,8], positive-unlabeled learning [9,10,11,12], unlabeled-unlabeled learning [13,14], noisy-label learning [15,16,17,18], complementary and partial-label learning [19,20,21,22,23,24,25,26,27], similarity-based learning [28,29,30], and positive-confidence learning [31].…”

Section: Introductionmentioning

confidence: 99%

Multi-Class Classification from Single-Class Data with Confidences

Cao,

Feng,

Shu

et al. 2021

Preprint

View full text Add to dashboard Cite

Can we learn a multi-class classifier from only data of a single class? We show that without any assumptions on the loss functions, models, and optimizers, we can successfully learn a multi-class classifier from only data of a single class with a rigorous consistency guarantee when confidences (i.e., the class-posterior probabilities for all the classes) are available. Specifically, we propose an empirical risk minimization framework that is loss-/model-/optimizer-independent. Instead of constructing a boundary between the given class and other classes, our method can conduct discriminative classification between all the classes even if no data from the other classes are provided. We further theoretically and experimentally show that our method can be Bayes-consistent with a simple modification even if the provided confidences are highly noisy. Then, we provide an extension of our method for the case where data from a subset of all the classes are available. Experimental results demonstrate the effectiveness of our methods.

show abstract

RECORD: Resource Constrained Semi-Supervised Learning under Distribution Shift

Cited by 14 publications

References 17 publications

Learning from Similarity-Confidence Data

Learning from Similarity-Confidence Data

Robust Deep Semi-Supervised Learning: A Brief Introduction

Multi-Class Classification from Single-Class Data with Confidences

Contact Info

Product

Resources

About