ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

Berthelot, David; Carlini, Nicholas; Cubuk, Ekin D.; Kurakin, Alexey; Sohn, Kihyuk; Zhang, Han; Raffel, Colin

doi:10.48550/arxiv.1911.09785

Cited by 138 publications

(291 citation statements)

References 0 publications

Supporting

Mentioning

287

Contrasting

Unclassified

Order By: Relevance

“…(c) Other methods include adversarial training with GANs [74], [75], [76], [77], [78], [79], [80], entropy minimization [11], and hybrid methods [81], [82], [83] which use a combination of above methods along with additional regularization terms.…”

Section: Related Workmentioning

confidence: 99%

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Chaitanya¹,

Erdil²,

Karani³

et al. 2021

Preprint

View full text Add to dashboard Cite

Supervised deep learning-based methods yield accurate results for medical image segmentation. However, they require large labeled datasets for this, and obtaining them is a laborious task that requires clinical expertise. Semi/selfsupervised learning-based approaches address this limitation by exploiting unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images and achieve high performance in classification tasks on popular natural image datasets like ImageNet. In pixel-level prediction tasks such as segmentation, it is crucial to also learn good local level representations along with global representations to achieve better accuracy. However, the impact of the existing local contrastive loss-based methods remains limited for learning good local representations because similar and dissimilar local regions are defined based on random augmentations and spatial proximity; not based on the semantic label of local regions due to lack of large-scale expert annotations in the semi/self-supervised setting. In this paper, we propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information obtained from pseudo-labels of unlabeled images alongside limited annotated images. In particular, we define the proposed loss to encourage similar representations for the pixels that have the same pseudo-label/ label while being dissimilar to the representation of pixels with different pseudo-label/label in the dataset. We perform pseudo-label based self-training and train the network by jointly optimizing the proposed contrastive loss on both labeled and unlabeled sets and segmentation loss on only the limited labeled set. We evaluated the proposed approach on three public medical datasets of cardiac and prostate anatomies, and obtain high segmentation performance with a limited labeled set of one or two 3D volumes. Extensive comparisons with the state-of-the-art semi-supervised and data augmentation methods and concurrent contrastive learning methods demonstrate the substantial improvement achieved by the proposed method. The code is made publicly available at https://github.com/krishnabits001/pseudo_label_contrastive_training.

show abstract

Section: Related Workmentioning

confidence: 99%

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Chaitanya¹,

Erdil²,

Karani³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Semi-supervised learning (SSL) [29], [31], [33], [42]- [44] leverages unlabeled data to improve a model's performance when limited labeled data is provided, which alleviates the expensive labeling process efficiently. Some recently proposed semi-supervised learning methods, such as MixMatch [43], FixMatch [31], and ReMixMatch [42], are based on augmentation viewpoints. MixMatch [43] uses low-entropy labels for data-augmented unlabeled instances and mixes labeled and unlabeled data for semi-supervised learning.…”

Section: Semi-supervised Learningmentioning

confidence: 99%

“…Fix-Match [31] generates pseudo labels using the model's predictions on weakly augmented unlabeled images. ReMix-Match [42] uses a weakly-augmented example to generate an artificial label and enforce consistency against stronglyaugmented examples. Semi-supervised domain adaptation has more information about some target labels compared with UDA, and some related works [45]- [49] have been proposed leveraging semi-supervised signals.…”

Section: Semi-supervised Learningmentioning

confidence: 99%

Unsupervised Domain Adaptation for Device-free Gesture Recognition

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

Device-free human gesture recognition with Radio Frequency (RF) signals has attained acclaim due to the omnipresence, privacy protection, and broad coverage nature of RF signals. However, neural network models trained for recognition with data collected from a specific domain suffer from significant performance degradation when applied to a new domain. To tackle this challenge, we propose an unsupervised domain adaptation framework for RF-based gesture recognition by making effective use of the unlabeled target domain data. Specifically, we apply pseudo-labeling and consistency regularization with elaborated design on target domain data to produce pseudo-label and align instance's feature of the target domain. Then, we design two data augmentation methods by randomly erasing the input data to enhance the robustness of the model. Furthermore, we apply a confidence control constraint to tackle the overconfidence problem. We conduct extensive experiments on a public Wi-Fi dataset and a public millimeter wave (mmWave) radar dataset. The experimental results demonstrate the superior effectiveness of the proposed framework.

show abstract

“…Its performance is comparable to that of supervised methods. Moreover, the augmentation idea in self-supervised learning is also adopted in some semi-supervised learning methods [24], [54], where augmentation anchoring is applied to replace the consistency regularization as it shows a stronger generalization ability.…”

Section: Self-supervised Learningmentioning

confidence: 99%

Adaptive Hierarchical Similarity Metric Learning with Noisy Labels

Yan¹,

Luo²,

Deng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep Metric Learning (DML) plays a critical role in various machine learning tasks. However, most existing deep metric learning methods with binary similarity are sensitive to noisy labels, which are widely present in real-world data. Since these noisy labels often cause a severe performance degradation, it is crucial to enhance the robustness and generalization ability of DML. In this paper, we propose an Adaptive Hierarchical Similarity Metric Learning method. It considers twonoiseinsensitive information, i.e., class-wise divergence and samplewise consistency. Specifically, class-wise divergence can effectively excavate richer similarity information beyond binary in modeling by taking advantage of Hyperbolic metric learning, while samplewise consistency can further improve the generalization ability of the model using contrastive augmentation. More importantly, we design an adaptive strategy to integrate this information in a unified view. It is noteworthy that the new method can be extended to any pair-based metric loss. Extensive experimental results on benchmark datasets demonstrate that our method achieves state-of-the-art performance compared with current deep metric learning approaches.

show abstract

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

Cited by 138 publications

References 0 publications

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Unsupervised Domain Adaptation for Device-free Gesture Recognition

Adaptive Hierarchical Similarity Metric Learning with Noisy Labels

Contact Info

Product

Resources

About