Impact of ImageNet Model Selection on Domain Adaptation

Zhang, Youshan; Davison, Brian D.

doi:10.1109/wacvw50321.2020.9096945

Cited by 34 publications

(23 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is obvious that pre-trained features from EfficientNetB7 have the smallest distance between two domains, which suggests the EfficientNetB7 is the best deep network compared to the other 16 networks. In addition, this observation is consistent with [36], that a better ImageNet model will produce better pre-trained features for UDA. Furthermore, we also list the computation time for feature extraction of each dataset.…”

Section: Resultssupporting

confidence: 87%

“…However, one disadvantage of feature extraction using pretrained networks is that it has lower performance than finetuning the same network since feature extraction is a single pass over the images. However, previous work [36] suggests that a better ImageNet model will produce better features for UDA. Therefore, we can extract features from a better ImageNet model to compensate for the lower performance of not fine-tuning.…”

Section: Pre-training Feature Representationmentioning

confidence: 95%

“…Recent deep networks frequently select ResNet50 as the backbone network for UDA [30,15]. Notably, other deep networks are not investigated, even though different Ima-geNet models affect the performance of UDA on traditional methods [36,40]. The impact of extracted pre-trained features on deep networks are not well explored.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Pre-trained Features and Recurrent Pseudo-Labeling in Unsupervised Domain Adaptation

Zhang

Davison

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Domain adaptation (DA) mitigates the domain shift problem when transferring knowledge from one annotated domain to another similar but different unlabeled domain. However, existing models often utilize one of the ImageNet models as the backbone without exploring others, and finetuning or retraining the backbone ImageNet model is also time-consuming. Moreover, pseudo-labeling has been used to improve the performance in the target domain, while how to generate confident pseudo labels and explicitly align domain distributions has not been well addressed. In this paper, we show how to efficiently opt for the best pre-trained features from seventeen well-known ImageNet models in unsupervised DA problems. In addition, we propose a recurrent pseudo-labeling model using the best pre-trained features (termed PRPL) to improve classification performance. To show the effectiveness of PRPL, we evaluate it on three benchmark datasets, Office+Caltech-10, Office-31, and Office-Home. Extensive experiments show that our model reduces computation time and boosts the mean accuracy to 98.1%, 92.4%, and 81.2%, respectively, substantially outperforming the state of the art.

show abstract

Section: Resultssupporting

confidence: 87%

Section: Pre-training Feature Representationmentioning

confidence: 95%

See 1 more Smart Citation

Efficient Pre-trained Features and Recurrent Pseudo-Labeling in Unsupervised Domain Adaptation

Zhang

Davison

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We clearly observe that CUDA outperforms several state-of-the-art methods that also use ResNet-50 [67] and even further surpasses by using ResNet-152 [67] encoder with CUDA. From our ablations study in Table 2, we show the effect of selection of ImageNet pre-trained ResNet-50 and ResNet-152 models on the domain adaptation with similar implications with the work [72].…”

Section: Office-31 Dataset Single-source Domain Adaptation Resultsmentioning

confidence: 53%

Contradistinguisher: A Vapnik's Imperative to Unsupervised Domain Adaptation

Balgi

Dukkipati

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

A complex combination of simultaneous supervised-unsupervised learning is believed to be the key to humans performing tasks seamlessly across multiple domains or tasks. This phenomenon of cross-domain learning has been very well studied in domain adaptation literature. Recent domain adaptation works rely on an indirect way of first aligning the source and target domain distributions and then train a classifier on the labeled source domain to classify the target domain. However, this approach has the main drawback that obtaining a near-perfect alignment of the domains in itself might be difficult/impossible (e.g., language domains).To address this, we follow Vapnik's imperative of statistical learning that states any desired problem should be solved in the most direct way rather than solving a more general intermediate task and propose a direct approach to domain adaptation that does not require domain alignment. We propose a model referred Contradistinguisher that learns contrastive features and whose objective is to jointly learn to contradistinguish the unlabeled target domain in an unsupervised way and classify in a supervised way on the source domain. We achieve the state-of-the-art on Office-31 and VisDA-2017 datasets in both single-source and multi-source settings. We also demonstrate that the contradistinguish loss improves the model performance by increasing the shape bias.

show abstract

“…We implement our approach using PyTorch with an Nvidia GeForce 1080 Ti GPU and extract features for the three datasets from a fine-tuned ResNet50 network [14]. The 1,000 features are then extracted from the last fully connected layer [15]. Parameters in domain distribution alignment are η = 0.1, λ = 10, and ρ = 10, which are fixed based on previous research [10], and τ = 0.31 is from [16].…”

Section: B Implementation Detailsmentioning

confidence: 99%

Correlated Adversarial Joint Discrepancy Adaptation Network

Zhang

Davison

2021

2021 International Conference on Content-Based Multimedia Indexing (CBMI)

Self Cite

View full text Add to dashboard Cite

Domain adaptation aims to mitigate the domain shift problem when transferring knowledge from one domain into another similar but different domain. However, most existing works rely on extracting marginal features without considering class labels. Moreover, some methods name their model as so-called unsupervised domain adaptation while tuning the parameters using the target domain label. To address these issues, we propose a novel approach called correlated adversarial joint discrepancy adaptation network (CAJNet), which minimizes the joint discrepancy of two domains and achieves competitive performance with tuning parameters using the correlated label. By training the joint features, we can align the marginal and conditional distributions between the two domains. In addition, we introduce a probability-based top-K correlated label (K-label), which is a powerful indicator of the target domain and effective metric to tune parameters to aid predictions. Extensive experiments on benchmark datasets demonstrate significant improvements in classification accuracy over the state of the art.

show abstract

Impact of ImageNet Model Selection on Domain Adaptation

Cited by 34 publications

References 33 publications

Efficient Pre-trained Features and Recurrent Pseudo-Labeling in Unsupervised Domain Adaptation

Efficient Pre-trained Features and Recurrent Pseudo-Labeling in Unsupervised Domain Adaptation

Contradistinguisher: A Vapnik's Imperative to Unsupervised Domain Adaptation

Correlated Adversarial Joint Discrepancy Adaptation Network

Contact Info

Product

Resources

About