A Practically Competitive and Provably Consistent Algorithm for Uplift Modeling

Abstract

Randomized experiments have been critical tools of decision making for decades. However, subjects can show significant heterogeneity in response to treatments in many important applications. Therefore it is not enough to simply know which treatment is optimal for the entire population. What we need is a model that correctly customize treatment assignment base on subject characteristics. The problem of constructing such models from randomized experiments data is known as Uplift Modeling in the literature. Many algorithms have been proposed for uplift modeling and some have generated promising results on various data sets. Yet little is known about the theoretical properties of these algorithms. In this paper, we propose a new tree-based ensemble algorithm for uplift modeling. Experiments show that our algorithm can achieve competitive results on both synthetic and industry-provided data. In addition, by properly tuning the "node size" parameter, our algorithm is proved to be consistent under mild regularity conditions. This is the first consistent algorithm for uplift modeling that we are aware of.

Bridging the Gap: A Systematic Benchmarking of Uplift Modeling and Heterogeneous Treatment Effects Methods

“…Other methods, however, perform only marginally worse. U-KL, as well as U-ED (Sołtys, Jaroszewicz, and Rzepakowski 2015), the BCF (Hahn, Murray, and Carvalho 2020), and CTS (Zhao, Fang, and Simchi-Levi 2017b) also perform very well and robustly in our study. Further, we show that the CIT (Su et al 2012) and DDP criterion (Hansotia and Rukstales 2002), initially suggested in the form of a CART and CHAID decision tree, respectively, also perform very well and robustly when we use these approaches in the form of a random forest.…”

“…Leveraging the incremental effect of a treatment on customer response—hereinafter referred to as the individual treatment effect (ITE)—to optimize targeting policies has been widely discussed in many research areas, including marketing (Ascarza 2018), economics (Athey and Imbens 2016), and data mining (Zhao, Fang, and Simchi-Levi 2017a). In the past few years, several researchers (Devriendt, Moldovan, and Verbeke 2018; Simester, Timoshenko, and Zoumpoulis 2019; Wager and Athey 2018) have written extensively about the incremental effect of a treatment and how to measure, identify, and leverage it.…”

A comparison of methods for model selection when estimating individual treatment effects

“…Modeling the conditional means is a valid approach, but many have noted that since the object of interest is the treatment effect we may be better off modeling it directly without appeal to the correctness of μ0 and μ1 . Approaches in this vein include Zhao et al [32], Athey et al [6], Powers et al [23], and Nie and Wager [22].…”