Balancing Transferability and Discriminability for Unsupervised Domain Adaptation

Huang, Jingke; Xiao, Ni; Zhang, Lei

doi:10.1109/tnnls.2022.3201623

Cited by 21 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…T σ(p v ) denotes the similarity of the i-th sample and the v-th sample with the soft-max operation on their model outputs p i , p v , γ iv is a threshold function used to determine whether the two samples belong to one class. The last class approaches preserves a memory bank to store the outputs of the historical models, thus constructing positive and negative sample pairs [122], [123]. HCID [122] uses the current model to encode the samples of the current batch as the query q t = M t (x q ), and then uses the historical model to encode the previously stored samples as keys…”

Section: Contrastive Learningmentioning

confidence: 99%

“…The last class approaches preserves a memory bank to store the outputs of the historical models, thus constructing positive and negative sample pairs [122], [123]. HCID [122] uses the current model to encode the samples of the current batch as the query q t = M t (x q ), and then uses the historical model to encode the previously stored samples as keys…”

Section: Contrastive Learningmentioning

confidence: 99%

See 1 more Smart Citation

Source-Free Domain Adaptation for Real-World Image Dehazing

Huang

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Over the past decade, domain adaptation has become a widely studied branch of transfer learning that aims to improve performance on target domains by leveraging knowledge from the source domain. Conventional domain adaptation methods often assume access to both source and target domain data simultaneously, which may not be feasible in real-world scenarios due to privacy and confidentiality concerns. As a result, the research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years, which only utilizes the source-trained model and unlabeled target data to adapt to the target domain. Despite the rapid explosion of SFDA work, yet there has no timely and comprehensive survey in the field. To fill this gap, we provide a comprehensive survey of recent advances in SFDA and organize them into a unified categorization scheme based on the framework of transfer learning. Instead of presenting each approach independently, we modularize several components of each method to more clearly illustrate their relationships and mechanics in light of the composite properties of each method. Furthermore, we compare the results of more than 30 representative SFDA methods on three popular classification benchmarks, namely Office-31, Office-home, and VisDA, to explore the effectiveness of various technical routes and the combination effects among them. Additionally, we briefly introduce the applications of SFDA and related fields. Drawing from our analysis of the challenges facing SFDA, we offer some insights into future research directions and potential settings.

show abstract

Section: Contrastive Learningmentioning

confidence: 99%

Section: Contrastive Learningmentioning

confidence: 99%

Source-Free Domain Adaptation for Real-World Image Dehazing

Huang

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…To alleviate the dependence on source domain data, which may be inaccessible due to privacy issues or storage overhead, sourcefree domain adaptation (SFDA) emerges which handles DA on target data without access to source data [12], [13], [14], [15], [16]. SFDA is often achieved through self-training [12], [17], selfsupervised learning [16], [18] or introducing prior knowledge [12] • X. Xu is with I2R, A-STAR. Y. Su, and K. Jia are with the School of Electronic and Information Engineering, South China University of Technology.…”

Section: Introductionmentioning

confidence: 99%

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training

Su¹,

Xu²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deploying models on target domain data subject to distribution shift requires adaptation. Test-time training (TTT) emerges as a solution to this adaptation under a realistic scenario where access to full source domain data is not available, and instant inference on the target domain is required. Despite many efforts into TTT, there is a confusion over the experimental settings, thus leading to unfair comparisons. In this work, we first revisit TTT assumptions and categorize TTT protocols by two key factors. Among the multiple protocols, we adopt a realistic sequential test-time training (sTTT) protocol, under which we develop a test-time anchored clustering (TTAC) approach to enable stronger test-time feature learning. TTAC discovers clusters in both source and target domains and matches the target clusters to the source ones to improve adaptation. When source domain information is strictly absent (i.e. source-free) we further develop an efficient method to infer source domain distributions for anchored clustering. Finally, self-training (ST) has demonstrated great success in learning from unlabeled data and we empirically figure out that applying ST alone to TTT is prone to confirmation bias. Therefore, a more effective TTT approach is introduced by regularizing self-training with anchored clustering, and the improved model is referred to as TTAC++. We demonstrate that, under all TTT protocols, TTAC++ consistently outperforms the state-of-the-art methods on five TTT datasets, including corrupted target domain, selected hard samples, synthetic-to-real adaptation and adversarially attacked target domain. We hope this work will provide a fair benchmarking of TTT methods, and future research should be compared within respective protocols.

show abstract

“…The second category is the image translation loss that generates source data with target-like styles and appearance via GANs [7,239,253] and spectrum matching [14,185]. The third category is the self-training loss that re-trains the network iteratively with pseudo-labeled target samples [7,13,14,141,183,184,254,255].…”

Section: Related Workmentioning

confidence: 99%

“…This section presents the proposed historical contrastive learning [254] 1 that memorizes the source hypothesis to make up for the absence of source data as illustrated in Fig. 5.2.…”

Section: Historical Contrastive Learningmentioning

confidence: 99%

Transductive transfer learning for visual recognition

Huang¹

View full text Add to dashboard Cite

has given invaluable advice on almost all aspects of my Ph.D. study. I would also express great gratitude towards all the lab-mates in MICL, CSL and SCALE Labs and all the members in Prof Lu's research group for the enthusiastic discussions and priceless support in the last four years. Dayan, Aoran and Pengdeng brought me a lot of precious memory in my Ph.D. journey. Thank Jingyi, Kaiwen, Zichen, Yun and Xueying for their hardworking while we collaborated on projects and researches. I enjoy the good works we have done and the team-working time.Thanks to all of my thesis committee members, Prof. Lin Guosheng, Prof. Liu Ziwei and Prof. Chen Change Loy for the time, kind suggestions and feedback for my qualification examination and thesis. You have all been very patient and kind throughout the whole process, and your encouragement and suggestions have helped refine my work greatly.Last but most importantly, I wish to express the deepest gratitude to my family for their constant and unconditional love and support. Thank you for being always there for me, for listening, for enlightening and for giving me the strength and courage to overcome difficulties and pursue greatness.xi

show abstract

Balancing Transferability and Discriminability for Unsupervised Domain Adaptation

Cited by 21 publications

References 33 publications

Source-Free Domain Adaptation for Real-World Image Dehazing

Source-Free Domain Adaptation for Real-World Image Dehazing

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training

Transductive transfer learning for visual recognition

Contact Info

Product

Resources

About