Adaptively Exploiting d-Separators with Causal Bandits

Abstract

Multi-armed bandit problems provide a framework to identify the optimal intervention over a sequence of repeated experiments. Without additional assumptions, minimax optimal performance (measured by cumulative regret) is well-understood. With access to additional observed variables that d-separate the intervention from the outcome (i.e., they are a d-separator), recent causal bandit algorithms provably incur less regret. However, in practice it is desirable to be agnostic to whether observed variables are a d-separator. Ideally, an algorithm should be adaptive; that is, perform nearly as well as an algorithm with oracle knowledge of the presence or absence of a d-separator. In this work, we formalize and study this notion of adaptivity, and provide a novel algorithm that simultaneously achieves (a) optimal regret when a d-separator is observed, improving on classical minimax algorithms, and (b) significantly smaller regret than recent causal bandit algorithms when the observed variables are not a d-separator. Crucially, our algorithm does not require any oracle knowledge of whether a d-separator is observed. We also generalize this adaptivity to other conditions, such as the front-door criterion.

Causal Structure Learning: a Combinatorial Perspective

“…By assuming that actions correspond to interventions in a known causal graph, the effects of different actions become related, allowing for better regret bounds [102,115]. If the causal graph is not assumed to be known, there is an additional exploration-exploitation tradeoff that needs to be taken into account, which has been considered in recent work [97,108,18]. Since certain parts of the causal graph might not be relevant to predicting the effect of an action on some reward, the reinforcement learning setting is another case in which targeted structure learning may be more efficient.…”

Causal Structure Learning: A Combinatorial Perspective

“…By assuming that actions correspond to interventions in a known causal graph, the effects of different actions become related, allowing for better regret bounds [102,116]. If the causal graph is not assumed to be known, there is an additional exploration-exploitation trade-off that needs to be taken into account, which has been considered in recent work [18,97,108]. Since certain parts of the causal graph might not be relevant to predicting the effect of an action on some reward, the reinforcement learning setting is another case in which targeted structure learning may be more efficient.…”