Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization

Akuzawa, Kei; Iwasawa, Yusuke; Matsuo, Yutaka

doi:10.1007/978-3-030-46147-8_19

Cited by 36 publications

(34 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important property of CF-Net is its ability to model continuous confounders (e.g., age), whereas most existing fair machine-learning methods 17,23,[51][52][53]25 are confined to binary or discrete confounders (e.g., gender). This improvement is achieved by our loss function based on squared correlation (see "Methods" section), which encourages statistical mean independence between the derived high-dimensional features and a scalar extraneous variable (in our case, a confounder).…”

Section: Discussionmentioning

confidence: 99%

Training confounder-free deep learning models for medical applications

2020

View full text Add to dashboard Cite

The presence of confounding effects (or biases) is one of the most critical challenges in using deep learning to advance discovery in medical imaging studies. Confounders affect the relationship between input data (e.g., brain MRIs) and output variables (e.g., diagnosis). Improper modeling of those relationships often results in spurious and biased associations. Traditional machine learning and statistical models minimize the impact of confounders by, for example, matching data sets, stratifying data, or residualizing imaging measurements. Alternative strategies are needed for state-of-the-art deep learning models that use end-to-end training to automatically extract informative features from large set of images. In this article, we introduce an end-to-end approach for deriving features invariant to confounding factors while accounting for intrinsic correlations between the confounder(s) and prediction outcome. The method does so by exploiting concepts from traditional statistical methods and recent fair machine learning schemes. We evaluate the method on predicting the diagnosis of HIV solely from Magnetic Resonance Images (MRIs), identifying morphological sex differences in adolescence from those of the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA), and determining the bone age from X-ray images of children. The results show that our method can accurately predict while reducing biases associated with confounders. The code is available at https://github.com/qingyuzhao/br-net.

show abstract

Section: Discussionmentioning

confidence: 99%

Training confounder-free deep learning models for medical applications

2020

View full text Add to dashboard Cite

show abstract

“…Nonetheless, many discrepancy reduction methods of UDA have been adapted to domain generalization for learning domain-invariant representations. For example, domain adversarial learning [37,50,3,40,2] with entropy regularization [66], MMD [36], and moment matching [41,19] have demonstrated effectiveness on domain generalization problems. See [67] for a more comprehensive survey.…”

Section: Related Workmentioning

confidence: 99%

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation

Berthelot¹,

Roelofs²,

Sohn³

et al. 2021

Preprint

View full text Add to dashboard Cite

We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one. With the goal of generality, we introduce AdaMatch, a method that unifies the tasks of unsupervised domain adaptation (UDA), semi-supervised learning (SSL), and semi-supervised domain adaptation (SSDA). In an extensive experimental study, we compare its behavior with respective state-of-the-art techniques from SSL, SSDA, and UDA on vision classification tasks. We find AdaMatch either matches or significantly exceeds the state-of-the-art in each case using the same hyper-parameters regardless of the dataset or task. For example, AdaMatch nearly doubles the accuracy compared to that of the prior state-of-the-art on the UDA task for DomainNet and even exceeds the accuracy of the prior state-ofthe-art obtained with pre-training by 6.4% when AdaMatch is trained completely from scratch. Furthermore, by providing AdaMatch with just one labeled example per class from the target domain (i.e., the SSDA setting), we increase the target accuracy by an additional 6.1%, and with 5 labeled examples, by 13.6%. 2 * equal contribution 2 All code to reproduce results will be open-sourced.Preprint. Under review.

show abstract

“…Carlucci et al [28] propose an adversarial approach combining domain adaptation and generalization while also doing domain mapping. Akuzawa et al [2] note the domain-invariance objective may compete with the discriminative objective and thus develop a method to find the most domain-invariant representation that does not hurt classification performance. Li et al [140] note that previous domain-invariant methods typically assume balanced classes and develop a method to handle changes in class proportions.…”

Section: Domain Generalizationmentioning

confidence: 99%

A Survey of Unsupervised Deep Domain Adaptation

Wilson

Cook

2020

ACM Trans. Intell. Syst. Technol.

636

307

View full text Add to dashboard Cite

Deep learning has produced state-of-the-art results for a variety of tasks. While such approaches for supervised learning have performed well, they assume that training and testing data are drawn from the same distribution, which may not always be the case. As a complement to this challenge, single-source unsupervised domain adaptation can handle situations where a network is trained on labeled data from a source domain and unlabeled data from a related but different target domain with the goal of performing well at test-time on the target domain. Many single-source and typically homogeneous unsupervised deep domain adaptation approaches have thus been developed, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially costly target data labels. This survey will compare these approaches by examining alternative methods, the unique and common elements, results, and theoretical insights. We follow this with a look at application areas and open research directions.

show abstract

Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization

Cited by 36 publications

References 15 publications

Training confounder-free deep learning models for medical applications

Training confounder-free deep learning models for medical applications

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation

A Survey of Unsupervised Deep Domain Adaptation

Contact Info

Product

Resources

About