SimMatch: Semi-supervised Learning with Similarity Matching

Zheng, Mingkai; You, Suya; Huang, Lang; Wang, Fei; Qian, Chen; Xu, Chang

doi:10.1109/cvpr52688.2022.01407

Cited by 116 publications

(49 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, since our method can be integrated with any variants based on FixMatch, we also use the state-of-the-art method FlexMatch [38] as the baseline and called the integrated method F ullF lex. The results demonstrate that our method can significantly boost the baseline models, including Fix-Match and FlexMatch for almost all datasets, and meanwhile surpasses the latest methods, e.g., NP-Match [30], SimMatch [41] and DoubleMatch [29]. Our method has the following advantages: 1) FullMatch brings considerable improvements compared with FixMatch, especially when the amount of labeled data is extremely limited.…”

Section: Resultsmentioning

confidence: 84%

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Chen¹,

Tan²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. The latest methods (e.g., FixMatch) use a combination of consistency regularization and pseudo-labeling to achieve remarkable successes. However, these methods all suffer from the waste of complicated examples since all pseudo-labels have to be selected by a high threshold to filter out noisy ones. Hence, the examples with ambiguous predictions will not contribute to the training phase. For better leveraging all unlabeled examples, we propose two novel techniques: Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL). EML incorporates the prediction distribution of non-target classes into the optimization objective to avoid competition with target class, and thus generating more high-confidence predictions for selecting pseudo-label. ANL introduces the additional negative pseudo-label for all unlabeled data to leverage low-confidence examples. It adaptively allocates this label by dynamically evaluating the top-k performance of the model. EML and ANL do not introduce any additional parameter and hyperparameter. We integrate these techniques with FixMatch, and develop a simple yet powerful framework called FullMatch. Extensive experiments on several common SSL benchmarks (CIFAR-10/100, SVHN, STL-10 and ImageNet) demonstrate that FullMatch exceeds FixMatch by a large margin. Integrated with FlexMatch (an advanced FixMatch-based framework), we achieve state-of-the-art performance. Source code is at https://github.com/megvii-research/FullMatch.

show abstract

Section: Resultsmentioning

confidence: 84%

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Chen¹,

Tan²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In the conventional image domain, Zheng et al have proposed a similar metric learning approach using softmax aggregation for image classification [31]. However, their work makes use of a pre-trained backbone, is semi-supervised, and does not provide all of the possibilities for feature selection, disentanglement, and alignment as does LatSim (see Equation 5).…”

Section: B Latent Similaritymentioning

confidence: 99%

Latent Similarity Identifies Important Functional Connections for Phenotype Prediction

Orlichenko

Zhang

et al. 2023

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

Objective: Endophenotypes like brain age and fluid intelligence are important biomarkers of disease status. However, brain imaging studies to identify these biomarkers often encounter limited numbers of subjects but high dimensional imaging features, hindering reproducibility. Therefore, we develop an interpretable, multivariate classification/regression algorithm, called Latent Similarity (LatSim), suitable for small sample size but high feature dimension datasets. Methods: LatSim combines metric learning with a kernel similarity function and softmax aggregation to identify task-related similarities between subjects. Inter-subject similarity is utilized to improve performance on three prediction tasks using multi-paradigm fMRI data. A greedy selection algorithm, made possible by LatSim's computational efficiency, is developed as an interpretability method. Results: LatSim achieved significantly higher predictive accuracy at small sample sizes on the Philadelphia Neurodevelopmental Cohort (PNC) dataset. Connections identified by LatSim gave superior discriminative power compared to those identified by other methods. We identified 4 functional brain networks enriched in connections for predicting brain age, sex, and intelligence. Conclusion:We find that most information for a predictive task comes from only a few (1-5) connections. Additionally, we find that the default mode network is over-represented in the top connections of all predictive tasks. Significance: We propose a novel algorithm for small sample, high feature dimension datasets and use it to identify connections in task fMRI data. Our work should lead to new insights in both algorithm design and neuroscience research. 1

show abstract

“…The effectiveness of FixMatch and UDA sparked a new wave of research trying to extend or improve these frameworks. For example, FlexMatch [40], Dash [34], SimMatch [42], CCSSL [35], DP-SSL [33], and DoubleMatch [31] all propose ways to improve the strategies for pseudo-labeling and consistency regularization of UDA and FixMatch.…”

Section: Semi-supervised Learningmentioning

confidence: 99%

Improving Open-Set Semi-Supervised Learning with Self-Supervision

Wallin¹,

Svensson²,

Kahl³

et al. 2023

Preprint

View full text Add to dashboard Cite

Open-set semi-supervised learning (OSSL) is a realistic setting of semi-supervised learning where the unlabeled training set contains classes that are not present in the labeled set. Many existing OSSL methods assume that these out-of-distribution data are harmful and put effort into excluding data from unknown classes from the training objective. In contrast, we propose an OSSL framework that facilitates learning from all unlabeled data through self-supervision. Additionally, we utilize an energy-based score to accurately recognize data belonging to the known classes, making our method well-suited for handling uncurated data in deployment. We show through extensive experimental evaluations on several datasets that our method shows overall unmatched robustness and performance in terms of closed-set accuracy and open-set recognition compared with state-of-the-art for OSSL. Our code will be released upon publication.

show abstract

SimMatch: Semi-supervised Learning with Similarity Matching

Cited by 116 publications

References 29 publications

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Latent Similarity Identifies Important Functional Connections for Phenotype Prediction

Improving Open-Set Semi-Supervised Learning with Self-Supervision

Contact Info

Product

Resources

About