Error asymmetry in causal and anticausal regression

Abstract

It is generally difficult to make any statements about the expected prediction error in an univariate setting without further knowledge about how the data were generated. Recent work showed that knowledge about the real underlying causal structure of a data generation process has implications for various machine learning settings. Assuming an additive noise and an independence between data generating mechanism and its input, we draw a novel connection between the intrinsic causal relationship of two variables and the expected prediction error. We formulate the theorem that the expected error of the true data generating function as prediction model is generally smaller when the effect is predicted from its cause and, on the contrary, greater when the cause is predicted from its effect. The theorem implies an asymmetry in the error depending on the prediction direction. This is further corroborated with empirical evaluations in artificial and real-world data sets.

Toward Causal Representation Learning

“…The study [220] empirically corroborated these predictions, thus establishing an intriguing bridge between the structure of learning problems and certain physical properties (cause-effect direction) of real-world data generating processes. It also led to a range of follow-up work [32], [78], [97], [114], [115], [152], [153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other tasks [105], [106], [114], [187], [191].…”

Towards Causal Representation Learning

“…The study [218] empirically corroborated these predictions, thus establishing an intriguing bridge between the structure of learning problems and certain physical properties (cause-effect direction) of real-world data generating processes. It also led to a range of follow-up work [279,266,280,77,114,281,32,96,263,243,195,152,156,153,167,204,115], complementing the studies of Bareinboim and Pearl [12,185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other tasks [189,192,105,104,115].…”

Causal versions of Maximum Entropy and Principle of Insufficient Reason

“…The idea is to distinguish cause and effect from their bivariate distribution, a task that cannot be solved by causal discovery methods that rely on conditional independences only (Spirtes et al, 1993;Pearl, 2000), the new approaches employ statistical properties other than conditional independences. Although distinction of cause and effect from purely observational data is still challenging despite some progress (Kano and Shimizu, 2003;Zhang and Hyvärinen, 2009;Peters et al, 2017;Mooij et al, 2016;Marx and Vreeken, 2017;Blöbaum et al, 2017;Guyon et al, 2019;Janzing, 2019), the development has stimulated discussions in various directions regarding inferential asymmetries of cause and effect. On the one hand, the relation to the arrow of time in physics has been described in Allahverdyan and Janzing (2008); .…”