Generalizing to Unseen Domains: A Survey on Domain Generalization

Wang, Jindong; Lan, Cuiling; Liu, Chang; Ouyang, Yidong; Zeng, Wenjun; Qin, Tao

doi:10.1109/tkde.2022.3178128

Cited by 345 publications

(155 citation statements)

References 141 publications

Supporting

Mentioning

116

Contrasting

Order By: Relevance

“…• Some robustness evaluations (for surveys, see [54,170]) explicitly tackle the problem of distribution shifts, rejecting the Assumptions of Test Data Validity without questioning the other assumptions we have identified.…”

Section: Discussionmentioning

confidence: 99%

Evaluation Gaps in Machine Learning Practice

Hutchinson¹,

Rostamzadeh²,

Greer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Forming a reliable judgement of a machine learning (ML) model's appropriateness for an application ecosystem is critical for its responsible use, and requires considering a broad range of factors including harms, benefits, and responsibilities. In practice, however, evaluations of ML models frequently focus on only a narrow range of decontextualized predictive behaviours. We examine the evaluation gaps between the idealized breadth of evaluation concerns and the observed narrow focus of actual evaluations. Through an empirical study of papers from recent high-profile conferences in the Computer Vision and Natural Language Processing communities, we demonstrate a general focus on a handful of evaluation methods. By considering the metrics and test data distributions used in these methods, we draw attention to which properties of models are centered in the field, revealing the properties that are frequently neglected or sidelined during evaluation. By studying these properties, we demonstrate the machine learning discipline's implicit assumption of a range of commitments which have normative impacts; these include commitments to consequentialism, abstractability from context, the quantifiability of impacts, the limited role of model inputs in evaluation, and the equivalence of different failure modes. Shedding light on these assumptions enables us to question their appropriateness for ML system contexts, pointing the way towards more contextualized evaluation methodologies for robustly examining the trustworthiness of ML models.CCS Concepts: • Computing methodologies → Machine learning approaches.

show abstract

Section: Discussionmentioning

confidence: 99%

Evaluation Gaps in Machine Learning Practice

Hutchinson¹,

Rostamzadeh²,

Greer³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Distribution shift on classic data format (e.g. vision or texts) have been comprehensively investigated in some recent surveys (e.g., [119,167]). However, there are inadequate discussions on graph data.…”

Section: Reliability Against Distribution Shiftmentioning

confidence: 99%

A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection

Wu¹,

Li²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep graph learning has achieved remarkable progresses in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery. Despite these progresses, how to ensure various deep graph learning algorithms behave in a socially responsible manner and meet regulatory compliance requirements becomes an emerging problem, especially in risk-sensitive domains. Trustworthy graph learning (TwGL) aims to solve the above problems from a technical viewpoint. In contrast to conventional graph learning research which mainly cares about model performance, TwGL considers various reliability and safety aspects of the graph

show abstract

“…Out-of-distribution (OOD) generalization is one of the core problems in machine learning where the testing distribution is unknown and different from the training [35,36]. Some works [37][38][39] learn how to disentangle the informative and distinct components of the data, which is considered a good representation for out-of-distribution generalization.…”

Section: Out-of-distribution Generalizationmentioning

confidence: 99%

Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation

Bae¹,

Moon²,

Im³

2022

Preprint

View full text Add to dashboard Cite

Self-supervised monocular depth estimation has been widely studied recently. Most of the work has focused on improving performance on benchmark datasets, such as KITTI, but has offered a few experiments on generalization performance. In this paper, we investigate the backbone networks (e.g. CNNs, Transformers, and CNN-Transformer hybrid models) toward the generalization of monocular depth estimation. We first evaluate state-of-the-art models on diverse public datasets, which have never been seen during the network training. Next, we investigate the effects of texture-biased and shape-biased representations using the various texture-shifted datasets that we generated. We observe that Transformers exhibit a strong shape bias and CNNs do a strong texture-bias. We also find that shape-biased models show better generalization performance for monocular depth estimation compared to texture-biased models. Based on these observations, we newly design a CNN-Transformer hybrid network with a multi-level adaptive feature fusion module, called MonoFormer. The design intuition behind MonoFormer is to increase shape bias by employing Transformers while compensating for the weak locality bias of Transformers by adaptively fusing multi-level representations. Extensive experiments show that the proposed method achieves state-of-the-art performance with various public datasets. Our method also shows the best generalization ability among the competitive methods.Preprint. Under review.

show abstract

Generalizing to Unseen Domains: A Survey on Domain Generalization

Cited by 345 publications

References 141 publications

Evaluation Gaps in Machine Learning Practice

Evaluation Gaps in Machine Learning Practice

A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection

Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation

Contact Info

Product

Resources

About