Uplift-based evaluation and optimization of recommenders

Sato, Masahiro; Singh, Janmajay; Takemori, Sho; Sonoda, Takashi; Zhang, Qian; Ohkuma, Tomoko

doi:10.1145/3298689.3347018

Cited by 27 publications

(38 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Dunnhumby dataset includes purchase and promotion logs of 2,500 users at a retailer for 93 weeks. Following [29], the items featured in the weekly mailer are considered as recommendations. To ensure reliable estimate of purchase probabilities in the next step, we filtered the dataset according to the following conditions: users with at least 10 weeks of purchase logs, items with at least 10 weeks of purchase logs, and items with both treatment and control conditions.…”

Section: Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

Unbiased Learning for the Causal Effect of Recommendation

Sato

Takemori

Singh

et al. 2020

Fourteenth ACM Conference on Recommender Systems

Self Cite

View full text Add to dashboard Cite

Increasing users' positive interactions, such as purchases or clicks, is an important objective of recommender systems. Recommenders typically aim to select items that users will interact with. If the recommended items are purchased, an increase in sales is expected. However, the items could have been purchased even without recommendation. Thus, we want to recommend items that results in purchases caused by recommendation. This can be formulated as a ranking problem in terms of the causal effect. Despite its importance, this problem has not been well explored in the related research. It is challenging because the ground truth of causal effect is unobservable, and estimating the causal effect is prone to the bias arising from currently deployed recommenders. This paper proposes an unbiased learning framework for the causal effect of recommendation. Based on the inverse propensity scoring technique, the proposed framework first constructs unbiased estimators for ranking metrics. Then, it conducts empirical risk minimization on the estimators with propensity capping, which reduces variance under finite training samples. Based on the framework, we develop an unbiased learning method for the causal effect extension of a ranking metric. We theoretically analyze the unbiasedness of the proposed method and empirically demonstrate that the proposed method outperforms other biased learning methods in various settings. CCS Concepts: • Information systems → Recommender systems; • Computing methodologies → Learning from implicit feedback.

show abstract

Section: Experiments Setupmentioning

confidence: 99%

“…Previous work[29] describes the propensity estimation in the experiment section 3. The codes are available on arXiv as ancillary files 4.…”

mentioning

confidence: 99%

Unbiased Learning for the Causal Effect of Recommendation

Sato

Takemori

Singh

et al. 2020

Fourteenth ACM Conference on Recommender Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both methods predict purchase probabilities with and without recommendations and rank items by the difference of these probabilities. Another strategy is to directly optimize ranking models for the causal effect [38,39]. ULRMF and ULBPR [38] are heuristic pointwise and pairwise learning methods inspired by the label transformation [15,21] in uplift modeling [31,8].…”

Section: Related Workmentioning

confidence: 99%

“…Few works targeting recommendation causal effect exist [4,37,38,39], and it is largely an unexplored area of research. Among them, a recent work [39] employed IPS method [26] in causal inference field, and developed unbiased learning-torank methods.…”

Section: Introductionmentioning

confidence: 99%

Causality-Aware Neighborhood Methods for Recommender Systems

Sato¹,

Takemori²,

Singh³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The business objectives of recommenders, such as increasing sales, are aligned with the causal effect of recommendations. Previous recommenders targeting for the causal effect employ the inverse propensity scoring (IPS) in causal inference. However, IPS is prone to suffer from high variance. The matching estimator is another representative method in causal inference field. It does not use propensity and hence free from the above variance problem. In this work, we unify traditional neighborhood recommendation methods with the matching estimator, and develop robust ranking methods for the causal effect of recommendations. Our experiments demonstrate that the proposed methods outperform various baselines in ranking metrics for the causal effect. The results suggest that the proposed methods can achieve more sales and user engagement than previous recommenders.

show abstract

“…The heterogenous treatment effect describes the effect variability due to participants' characteristics and is widely utilized in the contexts of personalized medicine, policy design and customized marketing recommendation (Kent et al, 2018;Imai and Strauss, 2011;Sato et al, 2019). In †Contributed equally.…”

Section: Introductionmentioning

confidence: 99%

Propensity score regression for causal inference with treatment heterogeneity

Wu¹,

Han²,

Tong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding how treatment effects vary on individual characteristics is critical in the contexts of personalized medicine, personalized advertising and policy design. When the characteristics are of practical interest are only a subset of full covariate, non-parametric estimation is often desirable; but few methods are available due to the computational difficult. Existing nonparametric methods such as the inverse probability weighting methods have limitations that hinder their use in many practical settings where the values of propensity scores are close to 0 or 1. We propose the propensity score regression (PSR) that allows the non-parametric estimation of the heterogeneous treatment effects in a wide context. PSR includes two non-parametric regressions in turn, where it first regresses on the propensity scores together with the characteristics of interest, to obtain an intermediate estimate; and then, regress the intermediate estimates on the characteristics of interest only. By including propensity scores as regressors in the non-parametric manner, PSR is capable of substantially easing the computational difficulty while remain (locally) insensitive to any value of propensity scores. We present several appealing properties of PSR, including the consistency and asymptotical normality, and in particular the existence of an explicit variance estimator, from which the analytical behaviour of PSR and its precision can be assessed. Simulation studies indicate that PSR outperform existing methods in varying settings with extreme values of propensity scores. We apply our method to the national 2009 flu survey (NHFS) data to investigate the effects of seasonal influenza vaccination and having paid sick leave across different age groups.

show abstract

Uplift-based evaluation and optimization of recommenders

Cited by 27 publications

References 38 publications

Unbiased Learning for the Causal Effect of Recommendation

Unbiased Learning for the Causal Effect of Recommendation

Causality-Aware Neighborhood Methods for Recommender Systems

Propensity score regression for causal inference with treatment heterogeneity

Contact Info

Product

Resources

About