Domain generalization via optimal transport with metric similarity learning

Zhou, Fan; Jiang, Zhuqing; Shui, Changjian; Wang, Boyu; Chaib-draa, Brahim

doi:10.1016/j.neucom.2020.09.091

Cited by 42 publications

(30 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the model generalization ability over out-of-distribution test samples, some work focuses on aligning feature representations across different domains. The minimization of feature discrepancy can be conducted over various distance metrics, including second-order statistics (Sun & Saenko, 2016), maximum mean discrepancy (Tzeng et al, 2014) and Wasserstein distance (Zhou et al, 2021b), or measured by adversarial networks (Ganin et al, 2016). Others apply data augmentation to generate new samples or domains to promote the consistency of feature representations, such as Mixup across existing domains (Xu et al, 2020;Yan et al, 2020b), or in an adversarial manner (Zhao et al, 2020;Qiao et al, 2020).…”

Section: General Methods For Ood and Noisy Labelsmentioning

confidence: 99%

“…To help accelerate research by focusing community attention and simplifying systematic comparisons between data collection and implementation method, we present DrugOOD, a systematic OOD dataset curator and benchmark for AI-aided drug discovery which comes with an open-source Python package that fully automates the data curation process and OOD benchmarking process. We focus on the most challenging OOD setting: domain generalization (Zhou et al, 2021b) problem in AI-aided drug discovery, though DrugOOD can be easily adapted to other OOD settings, such as subpopulation shift (Koh et al, 2021) and domain adaptation (Zhuang et al, 2020). Our dataset is also the first AIDD dataset curator with realistic noise annotations, that can serve as an important testbed for the setting of learning under noise.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

Ji¹,

Zhang²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

AI-aided drug discovery (AIDD) is gaining increasing popularity due to its promise of making the search for new pharmaceuticals quicker, cheaper and more efficient. In spite of its extensive use in many fields, such as ADMET prediction, virtual screening, protein folding and generative chemistry, little has been explored in terms of the out-ofdistribution (OOD) learning problem with noise, which is inevitable in real world AIDD applications.In this work, we present DrugOOD 1 , a systematic OOD dataset curator and benchmark for AI-aided drug discovery, which comes with an open-source Python package that fully automates the data curation and OOD benchmarking processes. We focus on one of the most crucial problems in AIDD: drug target binding affinity prediction, which involves both macromolecule (protein target) and small-molecule (drug compound). In contrast to only providing fixed datasets, DrugOOD offers automated dataset curator with user-friendly customization scripts, rich domain annotations aligned with biochemistry knowledge, realistic noise annotations and rigorous benchmarking of state-of-the-art OOD algorithms. Since the molecular data is often modeled as irregular graphs using graph neural network (GNN) backbones, DrugOOD also serves as a valuable testbed for graph OOD learning problems. Extensive empirical studies have shown a significant performance gap between in-distribution and out-of-distribution experiments, which highlights the need to develop better schemes that can allow for OOD generalization under noise for AIDD.

show abstract

Section: General Methods For Ood and Noisy Labelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

Ji¹,

Zhang²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Explicit feature alignment. This line of works aligns the features across source domains to learn domain-invariant representations through explicit feature distribution alignment [59,117,118], or feature normalization [119,120,121,57].…”

Section: Domain-invariant Representation-based Dgmentioning

confidence: 99%

“…Motiian et al [55] introduced a cross-domain contrastive loss for representation learning, where mapped domains are semantically aligned and yet maximally separated. Some methods explicitly minimized the feature distribution divergence by minimizing the maximum mean discrepancy (MMD) [122,116,123,124], second order correlation [125,126,127], both mean and variance (moment matching) [118], Wasserstein distance [117], etc, of domains for either domain adaptation or domain generalization. Zhou et al [117] aligned the marginal distribution of different source domains via optimal transport by minimizing the Wasserstein distance to achieve domain-invariant feature space.…”

Section: Domain-invariant Representation-based Dgmentioning

confidence: 99%

Generalizing to Unseen Domains: A Survey on Domain Generalization

Wang

Lan

Liu

et al. 2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

194

View full text Add to dashboard Cite

Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increased interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. For years, great progress has been achieved. This paper presents the first review for recent advances in domain generalization. First, we provide a formal definition of domain generalization and discuss several related fields. Then, we categorize recent algorithms into three classes and present them in detail: data manipulation, representation learning, and learning strategy, each of which contains several popular algorithms. Third, we introduce the commonly used datasets and applications. Finally, we summarize existing literature and present some potential research topics for the future.

show abstract

“…To address the problem of DG, in practice, one usually parameterizes the hypothesis as composed of a representation function followed by a labeling function [Albuquerque et al, 2019, Dou et al, 2019, Li et al, 2018b, Zhou et al, 2021a. This approach has its roots in the seminal works of [Ben-David et al, 2007, 2010, where the upper bounds on the risk of unseen domain were derived for the simple but instructive binary classification setting.…”

Section: Introductionmentioning

confidence: 99%

Barycentric-alignment and invertibility for domain generalization

Lyu¹,

Nguyen²,

Ishwar³

et al. 2021

Preprint

View full text Add to dashboard Cite

For the Domain Generalization (DG) problem where the hypotheses are composed of a common representation function followed by a labeling function, we point out a shortcoming in existing approaches that fail to explicitly optimize for a term, appearing in a well-known and widely adopted upper bound to the risk on the unseen domain, that is dependent on the representation to be learned. To this end, we first derive a novel upper bound to the prediction risk. We show that imposing a mild assumption on the representation to be learned, namely manifold restricted invertibility that is known to be satisfied by many datasets, is sufficient to deal with this issue. Further, unlike existing approaches, our novel upper bound doesn't require the assumption of Lipschitzness of the loss function. In addition, the distributional discrepancy in the representation space is handled via the Wasserstein-2 barycenter cost. In this context, we creatively leverage old and recent transport inequalities, which link various optimal transport metrics, in particular the L 1 distance (also known as the total variation distance) and the Wasserstein-2 distances, with the Kullback-Liebler divergence. These analyses and insights motivate a new representation learning cost for DG that additively balances three competing objectives: 1) minimizing classification error across seen domains via cross entropy, 2) enforcing domain-invariance in the representation space via the Wasserstein-2 barycenter cost, and 3) promoting non-degenerate, nearly-invertible representation via one of two mechanisms, viz., an autoencoder-based reconstruction loss or a mutual information loss. It is to be noted that the proposed algorithms completely bypass the use of any adversarial training mechanism that is typical of many current domain generalization approaches. Simulation results on several standard datasets demonstrate superior performance compared to several well-known DG algorithms.

show abstract

Domain generalization via optimal transport with metric similarity learning

Cited by 42 publications

References 17 publications

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

Generalizing to Unseen Domains: A Survey on Domain Generalization

Barycentric-alignment and invertibility for domain generalization

Contact Info

Product

Resources

About