Crafting Better Contrastive Views for Siamese Representation Learning

Peng, Xiangyu; Wang, Kai; Zhu, Zheng; Wang, Mang; You, Yang

doi:10.48550/arxiv.2202.03278

Cited by 10 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also not logical to recover the token of the main object that is masked by the token of the unmasked background and other objects at this point. Therefore, we introduce ContrastiveCrop [24] to perform data augmentation on the input images in MIM. Since contrastive learning draws the positive case samples closer and pushes the negative case samples farther in the high-dimensional space.…”

Section: Data Augmentationmentioning

confidence: 99%

“…When the proportion of primary objects is relatively small or the unmasked tokens are mainly primary objects, the reconstruction of background or other objects is not meaningful for the encoder to learn the primary object features. Also, to perform the contrastive learning task better, we introduce ContrastiveCrop [24] to process the input images. It helps to crop the primary object area by positioning the primary object with the semantic information extracted by the encoder, thus minimizing the interference of background and secondary objects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improvements to Self-Supervised Representation Learning for Masked Image Modeling

Mao¹,

Yin²,

Chang³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper explores improvements to the masked image modeling (MIM) paradigm. The MIM paradigm enables the model to learn the main object features of the image by masking the input image and predicting the masked part by the unmasked part. We found the following three main directions for MIM to be improved. First, since both encoders and decoders contribute to representation learning, MIM uses only encoders for downstream tasks, which ignores the impact of decoders on representation learning. Although the MIM paradigm already employs small decoders with asymmetric structures, we believe that continued reduction of decoder parameters is beneficial to improve the representational learning capability of the encoder. Second, MIM solves the image prediction task by training the encoder and decoder together, and does not design a separate task for the encoder. To further enhance the performance of the encoder when performing downstream tasks, we designed the encoder for the tasks of comparative learning and token position prediction. Third, since the input image may contain background and other objects, and the proportion of each object in the image varies, reconstructing the tokens related to the background or to other objects is not meaningful for MIM to understand the main object representations. Therefore we use ContrastiveCrop to crop the input image so that the input image contains as much as possible only the main objects. Based on the above three improvements to MIM, we propose a new model, Contrastive Masked AutoEncoders (CMAE). We achieved a Top-1 accuracy of 65.84% on tinyimagenet using the ViT-B backbone, which is +2.89 outperforming the MAE of competing methods when all conditions are equal. Code will be made available.

show abstract

Section: Data Augmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improvements to Self-Supervised Representation Learning for Masked Image Modeling

Mao¹,

Yin²,

Chang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Nonetheless, the widely-used attention-guided Center Zoom mechanism only zooms in on the whole suspicious area, which increases the difficulties of feature decomposition and the probability of ignoring subtle regions [18], [43]. We delicately designed our CMZ strategy for reducing the intersection of the local-branch inputs inspired by Contrastive Crop [44], which is proposed first in self-supervised learning (SSL) for data augmentation.…”

Section: B Weakly-supervised Lesion Localization Modulementioning

confidence: 99%

CDNet: Contrastive Disentangled Network for Fine-Grained Image Categorization of Ocular B-Scan Ultrasound

Dan¹,

Li²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Precise and rapid categorization of images in the Bscan ultrasound modality is vital for diagnosing ocular diseases. Nevertheless, distinguishing various diseases in ultrasound still challenges experienced ophthalmologists. Thus a novel contrastive disentangled network (CDNet) is developed in this work, aiming to tackle the fine-grained image categorization (FGIC) challenges of ocular abnormalities in ultrasound images, including intraocular tumor (IOT), retinal detachment (RD), posterior scleral staphyloma (PSS), and vitreous hemorrhage (VH). Three essential components of CDNet are the weaklysupervised lesion localization module (WSLL), contrastive multizoom (CMZ) strategy, and hyperspherical contrastive disentangled loss (HCD-Loss), respectively. These components facilitate feature disentanglement for fine-grained recognition in both the input and output aspects. The proposed CDNet is validated on our ZJU Ocular Ultrasound Dataset (ZJUOUSD), consisting of 5213 samples. Furthermore, the generalization ability of CDNet is validated on two public and widely-used chest X-ray FGIC benchmarks. Quantitative and qualitative results demonstrate the efficacy of our proposed CDNet, which achieves state-ofthe-art performance in the FGIC task. Code is available at: https://github.com/ZeroOneGame/CDNet-for-OUS-FGIC.

show abstract

“…In [14] the authors discuss challenges for multi-factor datasets in the context of natural images and propose using a subset of labels during pretraining to selectively tune for desirable factors. Another recent work [28] proposes initializing views randomly but using approximate heatmaps from higher level convolutional layers as guidance for curating better views adding a little computation overhead.…”

Section: Related Workmentioning

confidence: 99%

From Labels to Priors in Capsule Endoscopy: A Prior Guided Approach for Improving Generalization with Few Labels

Vats¹,

Mohammed²,

Pedersen³

2022

Preprint

View full text Add to dashboard Cite

The lack of generalizability of deep learning approaches for the automated diagnosis of pathologies in Wireless Capsule Endoscopy (WCE) has prevented any significant advantages from trickling down to real clinical practices. As a result, disease management using WCE continues to depend on exhaustive manual investigations by medical experts. This explains its limited use despite several advantages. Prior works have considered using higher quality and quantity of labels as a way of tackling the lack of generalization, however this is hardly scalable considering pathology diversity not to mention that labeling large datasets encumbers the medical staff additionally. We propose using freely available domain knowledge as priors to learn more robust and generalizable representations. We experimentally show that domain priors can benefit representations by acting in proxy of labels, thereby significantly reducing the labeling requirement while still enabling fully unsupervised yet pathology-aware learning. We use the contrastive objective along with prior-guided views during pretraining, where the view choices inspire sensitivity to pathological information. Extensive experiments on three datasets show that our method performs better than (or closes gap with) the state-of-the-art in the domain, establishing a new benchmark in pathology classification and cross-dataset generalization, as well as scaling to unseen pathology categories.

show abstract

Crafting Better Contrastive Views for Siamese Representation Learning

Cited by 10 publications

References 37 publications

Improvements to Self-Supervised Representation Learning for Masked Image Modeling

Improvements to Self-Supervised Representation Learning for Masked Image Modeling

CDNet: Contrastive Disentangled Network for Fine-Grained Image Categorization of Ocular B-Scan Ultrasound

From Labels to Priors in Capsule Endoscopy: A Prior Guided Approach for Improving Generalization with Few Labels

Contact Info

Product

Resources

About