Contrastive Domain Adaptation

Thota, Mamatha; Leontidis, Georgios

doi:10.1109/cvprw53098.2021.00250

Cited by 50 publications

(29 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior works have explored the application of self-supervision to UDA. However, to the best of our knowledge they have all used domain distance either as part of their objective (Kang et al, 2019;Wang et al, 2021;Thota & Leontidis, 2021) or for model selection (Sun et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

Shen¹,

Jones²,

Kumar³

et al. 2022

Preprint

View full text Add to dashboard Cite

We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e.g., photographs ) and unlabeled data from a target domain (e.g., sketches) are used to learn a classifier for the target domain. Conventional UDA methods (e.g., domain adversarial training) learn domain-invariant features to improve generalization to the target domain. In this paper, we show that contrastive pre-training, which learns features on unlabeled source and target data and then fine-tunes on labeled source data, is competitive with strong UDA methods. However, we find that contrastive pre-training does not learn domain-invariant features, diverging from conventional UDA intuitions. We show theoretically that contrastive pre-training can learn features that vary subtantially across domains but still generalize to the target domain, by disentangling domain and class information. Our results suggest that domain invariance is not necessary for UDA . We empirically validate our theory on benchmark vision datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

Shen¹,

Jones²,

Kumar³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Representative techniques include latent distribution alignment between the source and target domains (Tzeng et al 2017;Hoffman et al 2018;Long et al 2017). Contrastive learning is used to extract discriminative features between classes (Kang et al 2019;Thota and Leontidis 2021), and the memory module is used to augment target features using incremental information (Asghar et al 2020;Zheng and Yang 2019;Liu et al 2020). A long-standing problem in domain adaptation is negative transfer, which refers to the abnormal scenarios when the source domain data causes reduced learning performance in the target domain due to a large discrepancy in data distributions (Wang et al 2019;Zhang et al 2020).…”

Section: Domain Adaptationmentioning

confidence: 99%

Knowledge Sharing via Domain Adaptation in Customs Fraud Detection

Park

Kim

Cha

2022

AAAI

View full text Add to dashboard Cite

Knowledge of the changing traffic is critical in risk management. Customs offices worldwide have traditionally relied on local resources to accumulate such knowledge and detect tax frauds. This naturally poses countries with weak infrastructure to become tax havens of potentially illicit trades. The current paper proposes DAS, a memory bank platform to facilitate knowledge sharing across multi-national customs administrations to support each other. We propose a domain adaptation method to share transferable knowledge of frauds as prototypes while safeguarding the local trade information. Data encompassing over 8 million import declarations have been used to test the feasibility of this new system, which shows that participating countries may benefit up to 2-11 times in fraud detection with the help of shared knowledge. We discuss implications for substantial tax revenue potential and strengthened policy against illicit trades.

show abstract

“…According to (Thota & Leontidis, 2021) CL has become a key approach for unsupervised learning tasks with unlabeled datasets (Chen et al, 2020; K. Jaiswal et al, 2020). Broadly, one sample from the unlabeled dataset is taken as a so-called anchor and a strongly augmented version of this sample is considered a positive sample.…”

Section: Contrastive Learningmentioning

confidence: 99%

“…(Jaiswal et al, 2020;Shen et al, 2022) Here, train and test samples are from the same dataset and distribution. A major problem of the contrastive self-supervised architecture is the occurrence of false negatives in the negative samples (Thota & Leontidis, 2021).…”

Section: Contrastive Learningmentioning

confidence: 99%

“…(Thota & Leontidis, 2021) follow an entirely unsupervised approach, by training only with unlabeled data. However, so far these applications remain largely underexplored in research (Thota & Leontidis, 2021). Furthermore, in the technical domains considered in this paper, different boundary conditions apply: Besides a labeled source-domain dataset, there are samples of the non-defective class of the target domain given in training.…”

Section: Contrastive Learningmentioning

confidence: 99%

See 1 more Smart Citation

Cross-domain Transfer of Defect Features in Technical Domains based on Partial Target Data

Schlagenhauf¹,

Scheurenbrand²

2022

Preprint

View full text Add to dashboard Cite

A common challenge in real-world classification scenarios with sequentially appending target domain data is insufficient training datasets during the training phase. Therefore, conventional deep learning and transfer learning classifiers are not applicable especially when individual classes are not represented or are severely underrepresented at the outset. Domain Generalization approaches reach their limits when domain shifts become too large, making them occasionally unsuitable as well. In many (technical) domains, however, it is only the defect/ worn/ reject classes that are insufficiently represented, while the non-defect class is often available from the beginning. The proposed classification approach addresses such conditions and is based on a CNN encoder. Following a contrastive learning approach, it is trained with a modified triplet loss function using two datasets: Besides the non-defective target domain class (= 1st dataset), a state-of-the-art labeled source domain dataset that contains highly related classes (e.g., a related manufacturing error or wear defect) but originates from a (highly) different domain (e.g., different product, material, or appearance) (= 2nd dataset) is utilized. The approach learns the classification features from the source domain dataset while at the same time learning the differences between the source and the target domain in a single training step, aiming to transfer the relevant features to the target domain. The classifier becomes sensitive to the classification features and – by architecture – robust against the highly domain-specific context. The approach is benchmarked in a technical and a non-technical domain and shows convincing classification results. In particular, it is shown that the domain generalization capabilities and classification results are improved by the proposed architecture, allowing for larger domain shifts between source and target domains.

show abstract

Contrastive Domain Adaptation

Cited by 50 publications

References 25 publications

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

Knowledge Sharing via Domain Adaptation in Customs Fraud Detection

Cross-domain Transfer of Defect Features in Technical Domains based on Partial Target Data

Contact Info

Product

Resources

About