Robustness via Cross-Domain Ensembles

Abstract

We present a method for making neural network predictions robust to shifts from the training data distribution. The proposed method is based on making predictions via a diverse set of cues (called 'middle domains') and ensembling them into one strong prediction. The premise of the idea is that predictions made via different cues respond differently to a distribution shift, hence one should be able to merge them into one robust final prediction. We perform the merging in a straightforward but principled manner based on the uncertainty associated with each prediction. The evaluations are performed using multiple tasks and datasets (Taskonomy, Replica, ImageNet, CIFAR) under a wide range of adversarial and non-adversarial distribution shifts which demonstrate the proposed method is considerably more robust than its standard learning counterpart, conventional deep ensembles, and several other baselines.

Enhancing Pseudo Label Quality for Semi-supervised Domain-Generalized Medical Image Segmentation

“…Domain generalization (DG) trains the model with multiple source domains and generalizes it to unseen target domains. Existing DG methods can be categorized into three classes: representation learning (Zhou et al 2020), learning strategy design (Yeo, Kar, and Zamir 2021), and data manipulation (Tobin et al 2017). Representation learning methods mainly follow the idea of domain adaptation by learning domain-invariant features or explicitly feature alignment between domains.…”

Enhancing Pseudo Label Quality for Semi-Supervised Domain-Generalized Medical Image Segmentation

“…Domain generalization (DG) trains the model with multiple source domains and generalizes it to unseen target domains. Existing DG methods can be categorized into three classes: representation learning (Zhou et al 2020), learning strategy design (Yeo, Kar, and Zamir 2021), and data manipulation (Tobin et al 2017). Representation learning methods mainly follow the idea of domain adaptation by learning domain-invariant features or explicitly feature alignment between domains.…”