Posterior Re-calibration for Imbalanced Datasets

Tian, Junjiao; Liu, Yen‐Cheng; Glaser, Nathaniel; Hsu, Yen-Chang; Kira, Zsolt

doi:10.48550/arxiv.2010.11820

Cited by 3 publications

(2 citation statements)

References 12 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, Jamal et al [2020], improved the performance on the rare examples by explicitly encourages transfer learning. Re-calibration methods adjust the logits of the outputs with re-weighting [Tian et al, 2020, Menon et al, 2021.…”

Section: Supervised Learning With Dataset Imbalancementioning

confidence: 99%

“…The performance of vanilla supervised methods degrades significantly on class-imbalanced datasets [Cui et al, 2019, Cao et al, 2019, Buda et al, 2018, posing challenges to practical applications such as instance segmentation and depth estimation . Many recent works address this issue with various regularization and re-weighting/re-sampling techniques [Ando and Huang, 2017, Wang et al, 2017b, Jamal et al, 2020, Cui et al, 2019, Cao et al, 2019, Tian et al, 2020, Hong et al, 2021.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self-supervised Learning is More Robust to Dataset Imbalance

Liu¹,

HaoChen²,

Gaidon³

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-supervised learning (SSL) is a scalable way to learn general visual representations since it learns without labels. However, large-scale unlabeled datasets in the wild often have long-tailed label distributions, where we know little about the behavior of SSL. In this work, we systematically investigate self-supervised learning under dataset imbalance. First, we find out via extensive experiments that off-the-shelf selfsupervised representations are already more robust to class imbalance than supervised representations. The performance gap between balanced and imbalanced pre-training with SSL is significantly smaller than the gap with supervised learning, across sample sizes, for both in-domain and, especially, out-ofdomain evaluation. Second, towards understanding the robustness of SSL, we hypothesize that SSL learns richer features from frequent data: it may learn label-irrelevant-but-transferable features that help classify the rare classes and downstream tasks. In contrast, supervised learning has no incentive to learn features irrelevant to the labels from frequent examples. We validate this hypothesis with semi-synthetic experiments and theoretical analyses on a simplified setting. Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples.

show abstract

Section: Supervised Learning With Dataset Imbalancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Self-supervised Learning is More Robust to Dataset Imbalance

Liu¹,

HaoChen²,

Gaidon³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Disentangling Label Distribution for Long-tailed Visual Recognition

Hong¹,

Han²,

Choi³

et al. 2020

Preprint

View full text Add to dashboard Cite

The current evaluation protocol of long-tailed visual recognition trains the classification model on the longtailed source label distribution and evaluates its performance on the uniform target label distribution. Such protocol has questionable practicality since the target may also be long-tailed. Therefore, we formulate long-tailed visual recognition as a label shift problem where the target and source label distributions are different. One of the significant hurdles in dealing with the label shift problem is the entanglement between the source label distribution and the model prediction. In this paper, we focus on disentangling the source label distribution from the model prediction. We first introduce a simple baseline method that matches the target label distribution by post-processing the model prediction trained by the cross-entropy loss and the Softmax function. Although this method surpasses stateof-the-art methods on benchmark datasets, it can be further improved by directly disentangling the source label distribution from the model prediction in the training phase. Thus, we propose a novel method, LAbel distribution Dis-Entangling (LADE) loss based on the optimal bound of Donsker-Varadhan representation. LADE achieves state-ofthe-art performance on benchmark datasets such as CIFAR-100-LT, Places-LT, ImageNet-LT, and iNaturalist 2018. Moreover, LADE outperforms existing methods on various shifted target label distributions, showing the general adaptability of our proposed method.

show abstract

On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

Pan¹,

Zhang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Vanilla models for object detection and instance segmentation suffer from the heavy bias toward detecting frequent objects in the long-tailed setting. Existing methods address this issue mostly during training, e.g., by re-sampling or reweighting. In this paper, we investigate a largely overlooked approach -postprocessing calibration of confidence scores. We propose NORCAL, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size. We show that separately handling the background class and normalizing the scores over classes for each proposal are keys to achieving superior performance. On the LVIS dataset, NORCAL can effectively improve nearly all the baseline models not only on rare classes but also on common and frequent classes. Finally, we conduct extensive analysis and ablation studies to offer insights into various modeling choices and mechanisms of our approach. * Equal contributions Preprint. Under review.

show abstract

Posterior Re-calibration for Imbalanced Datasets

Cited by 3 publications

References 12 publications

Self-supervised Learning is More Robust to Dataset Imbalance

Self-supervised Learning is More Robust to Dataset Imbalance

Disentangling Label Distribution for Long-tailed Visual Recognition

On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

Contact Info

Product

Resources

About