Distributional anchor regression

Abstract

Prediction models often fail if train and test data do not stem from the same distribution. Out-of-distribution (OOD) generalization to unseen, perturbed test data is a desirable but difficult-to-achieve property for prediction models and in general requires strong assumptions on the data generating process (DGP). In a causally inspired perspective on OOD generalization, the test data arise from a specific class of interventions on exogenous random variables of the DGP, called anchors. Anchor regression models, introduced by Rothenhäusler et al. (J R Stat Soc Ser B 83(2):215–246, 2021. 10.1111/rssb.12398), protect against distributional shifts in the test data by employing causal regularization. However, so far anchor regression has only been used with a squared-error loss which is inapplicable to common responses such as censored continuous or ordinal data. Here, we propose a distributional version of anchor regression which generalizes the method to potentially censored responses with at least an ordered sample space. To this end, we combine a flexible class of parametric transformation models for distributional regression with an appropriate causal regularizer under a more general notion of residuals. In an exemplary application and several simulation scenarios we demonstrate the extent to which OOD generalization is possible.

Causal Regularization: On the trade-off between in-sample risk and out-of-sample risk guarantees

“…However, if it is not known, cross-validation is recommended albeit no proof was provided regarding its adequacy for the proposed loss. Follow up work extended the method to noisy instrumental variables [Oberst et al, 2021], and discrete and censored outcomes [Kook et al, 2021].…”

Distributional regression modeling via generalized additive models for location, scale, and shape: An overview through a data set from learning analytics

“…The idea of causal regularization for enhancing stability and better external validity has been extended to a certain class of distributional regression models (Kook et al, 2022 ). The file “causal‐regularization‐supplement” in the repository (see end of Section 6), features the application of causal‐regularized distributional regression to the OULAD data set to demonstrate improved worst‐case prediction and better external validity.…”

Evaluating Robustness to Dataset Shift via Parametric Robustness Sets

“…In some settings, such shifts may not represent plausible changes, as we demonstrate in in Appendix D, where (in a simplified lab-testing example) the worst-case subpopulation is one where healthy patients are always tested, and sick patients never tested. Prior work on robustness to parametric interventions has been restricted to linear causal models with additive shift interventions [Rothenhäusler et al, 2021, Oberst et al, 2021, Kook et al, 2022. Our work can be seen as extending those ideas to general non-linear causal models, where our focus is on evaluation rather than learning robust models.…”