SaR: Self-adaptive Refinement on Pseudo Labels for Multiclass-Imbalanced Semi-supervised Learning

Lai, Zhengfeng; Wang, Chao; Cheung, Susan; Chuah, Chen-Nee

doi:10.1109/cvprw56347.2022.00454

Cited by 8 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, seminal works (Cui et al, 2019;Ren et al, 2020) reweight the loss functions according to the sampling frequency of each class. Recent literature (Lai et al, 2022) enhances the robustness of SSL to long-tailed class imbalanced problems by designing weights in the unsupervised loss based on estimating the learning difficulty of each class. In contrast, several studies (Cao et al, 2019;Menon et al, 2020;Tan et al, 2020) have attempted to adjust the loss margins of each class.…”

Section: Loss Function For Long-tailed Learningmentioning

confidence: 99%

“…For the unsupervised loss function CEL, the weight parameter (l) increases linearly per epoch according to l = l u Â epoch epoch max , and the confidence threshold t is set to 0.95. As in previous works (Sohn et al, 2020;Lai et al, 2022), we employ an exponential moving average of model parameters to generate the final performance. We keep other hyper-parameters the same as the ImageNet experiments in FixMatch, except for those mentioned above.…”

Section: Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Ma,

Wei,

Zhu

et al. 2024

Front. Mar. Sci.

View full text Add to dashboard Cite

Aquatic biodiversity monitoring relies on species recognition from images. While deep learning (DL) streamlines the recognition process, the performance of these method is closely linked to the large-scale labeled datasets, necessitating manual processing with expert knowledge and consume substantial time, labor, and financial resources. Semi-supervised learning (SSL) offers a promising avenue to improve the performance of DL models by utilizing the extensive unlabeled samples. However, the complex collection environments and the long-tailed class imbalance of aquatic species make SSL difficult to implement effectively. To address these challenges in aquatic species recognition within the SSL scheme, we propose a Wavelet Fusion Network and the Consistency Equilibrium Loss function. The former mitigates the influence of data collection environment by fusing image information at different frequencies decomposed through wavelet transform. The latter improves the SSL scheme by refining the consistency loss function and adaptively adjusting the margin for each class. Extensive experiments are conducted on the large-scale FishNet dataset. As expected, our method improves the existing SSL scheme by up to 9.34% in overall classification accuracy. With the accumulation of image data, the improved SSL method with limited labeled data, shows the potential to advance species recognition for aquatic biodiversity monitoring and conservation.

show abstract

Section: Loss Function For Long-tailed Learningmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Ma,

Wei,

Zhu

et al. 2024

Front. Mar. Sci.

View full text Add to dashboard Cite

show abstract

“…generate unbiased and accurate pseudo-labels, such as resampling (Lee, Shin, and Kim 2021;Guo and Li 2022), re-weighting (Lai et al 2022), transfer learning (Fan et al 2022), and logit adjustment (Wang et al 2022a). However, these works usually assume a similar class distribution between labeled and unlabeled set and show inferior performances when such assumption is violated.…”

Section: Long-tailed Learningmentioning

confidence: 99%

“…Baselines We compare with several LTSSL algorithms published in top-conferences/journals in the past few years. These baseline algorithms include DARP (Kim et al 2020), CReST (Wei et al 2021) and its variant CReST+ (Wei et al 2021), ABC (Lee, Shin, and Kim 2021), DASO (Oh, Kim, and Kweon 2022), CoSSL (Fan et al 2022), SAW (Lai et al 2022), Adsh (Guo and Li 2022), DePL (Wang et al 2022a), RDA (Duan et al 2022), and ACR (Wei and Gan 2023). For a fair comparison, we test these baselines and our CPE algorithm on the widely-used codebase USB 1 .…”

Section: Experimental Settingmentioning

confidence: 99%

Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning

Ma,

Elezi,

Deng

et al. 2024

AAAI

View full text Add to dashboard Cite

We address the challenging problem of Long-Tailed Semi-Supervised Learning (LTSSL) where labeled data exhibit imbalanced class distribution and unlabeled data follow an unknown distribution. Unlike in balanced SSL, the generated pseudo-labels are skewed towards head classes, intensifying the training bias. Such a phenomenon is even amplified as more unlabeled data will be mislabeled as head classes when the class distribution of labeled and unlabeled datasets are mismatched. To solve this problem, we propose a novel method named ComPlementary Experts (CPE). Specifically, we train multiple experts to model various class distributions, each of them yielding high-quality pseudo-labels within one form of class distribution. Besides, we introduce Classwise Batch Normalization for CPE to avoid performance degradation caused by feature distribution mismatch between head and non-head classes. CPE achieves state-of-the-art performances on CIFAR-10-LT, CIFAR-100-LT, and STL-10-LT dataset benchmarks. For instance, on CIFAR-10-LT, CPE improves test accuracy by over >2.22% compared to baselines. Code is available at https://github.com/machengcheng2016/CPE-LTSSL.

show abstract

“…In recent semi-supervised learning methods, re-weighting is gradually gaining attention. Wei et al (2021) utilize the unlabeled data with pseudo labels according to an estimated class distribution, Kim et al (2020) develop a framework to refine class distribution, Lai et al (2022) use the effective number defined by Cui et al (2019) to produce adaptive weights.…”

Section: Related Workmentioning

confidence: 99%

Re-weighting Tokens: A Simple and Effective Active Learning Strategy for Named Entity Recognition

Luo,

Tan,

Nguyen

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Active learning, a widely adopted technique for enhancing machine learning models in text and image classification tasks with limited annotation resources, has received relatively little attention in the domain of Named Entity Recognition (NER). The challenge of data imbalance in NER has hindered the effectiveness of active learning, as sequence labellers lack sufficient learning signals. To address these challenges, this paper presents a novel reweightingbased active learning strategy that assigns dynamic smoothed weights to individual tokens. This adaptable strategy is compatible with various token-level acquisition functions and contributes to the development of robust active learners. Experimental results on multiple corpora demonstrate the substantial performance improvement achieved by incorporating our re-weighting strategy into existing acquisition functions, validating its practical efficacy.

show abstract

SaR: Self-adaptive Refinement on Pseudo Labels for Multiclass-Imbalanced Semi-supervised Learning

Cited by 8 publications

References 24 publications

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning

Re-weighting Tokens: A Simple and Effective Active Learning Strategy for Named Entity Recognition

Contact Info

Product

Resources

About