Confidence intervals for policy evaluation in adaptive experiments

Abstract

Adaptive experimental designs can dramatically improve efficiency in randomized trials. But with adaptively collected data, common estimators based on sample means and inverse propensity-weighted means can be biased or heavy-tailed. This poses statistical challenges, in particular when the experimenter would like to test hypotheses about parameters that were not targeted by the data-collection mechanism. In this paper, we present a class of test statistics that can handle these challenges. Our approach is to adaptively reweight the terms of an augmented inverse propensity-weighting estimator to control the contribution of each term to the estimator’s variance. This scheme reduces overall variance and yields an asymptotically normal test statistic. We validate the accuracy of the resulting estimates and their CIs in numerical experiments and show that our methods compare favorably to existing alternatives in terms of mean squared error, coverage, and CI size.

Barriers for Academic Data Science Research in the New Realm of Behavior Modification by Digital Platforms

“…Examples include data acquisition policies 28 , methods for handling missing values in predictive tasks 29 , adjusting for selfselection in large-scale human impact studies 30 , and identifying heterogeneous treatment effects 31 . Further examples include evaluating the impact of behavioral data size and richness on predictive ability 32 ; proposing and comparing model explainability approaches 33 and alternatives 34 ; developing sampling designs from networked data 35 and unbiased treatment effect estimators in adaptive experiments 36 ; and translating legal notions of discrimination into automated algorithmic solutions 37 .…”

“…The use of BMOD by platforms can interfere with research efforts by masking, changing, and even overriding effects of interest. For this reason, there is a growing niche of research focused on developing unbiased training and evaluation procedures using ideas drawn from causal inference 36,66 . As noted, in observational studies, users' BBD alone is insufficient for answering causal questions about human behavior and possible feedback loops between human behaviors and algorithm learning and actions.…”

An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan

“…But MAB heuristics pose a problem for an experimenter interested in estimating the effects of all treatments: if the experimenter is quickly convinced that a particular treatment is suboptimal, she should stop assigning it in the future. As a result, the experimenter might miss out on learning about the effectiveness of good, though suboptimal, policies; the resulting inferential problems are discussed in Hadad et al (2019).…”

“…One worry about adaptive experimental designs is that they lead to biased inference (see for instance Hadad et al 2019). Item 4 of Theorem 1 implies, however, that this is not the case for the Tempered Thompson Algorithm in large samples.…”

Behavioral Welfare Economics and Risk Preferences: A Bayesian Approach

“…Armed with posterior predictive estimates of the welfare gain or loss distribution for each subject and each choice, can we adaptively identify when to withdraw the insurance product from these persistent losers, and thereby avoid them incurring such large welfare losses? Important recent research by Caria et al [2020], Hadad et al [2020] and Kasy and Sautmann [2019] considers this general issue. The challenges are significant, from the effects on inference about confidence intervals, to the implications for optimal sampling intensity, to the weight to be given to multiple treatment arms, and so on.…”