Debiased Off-Policy Evaluation for Recommendation Systems

Narita, Yoshihiro; Yasui, Shota; Yata, Kohei

doi:10.1145/3460231.3474231

Cited by 9 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recommendation, causal learning has been used for tackling problem of the biases (e.g., position bias, popularity bias, selection bias etc.) [2,21,22,27] and fairness [9,18,20]. Many researchers focus on causal embedding for recommendation [3,15,33,45].…”

Section: Related Workmentioning

confidence: 99%

A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

Si,

Han,

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine-learning based recommender systems(RSs) has become an effective means to help people automatically discover their interests. Existing models often represent the rich information for recommendation, such as items, users, and contexts, as embedding vectors and leverage them to predict users' feedback. In the view of causal analysis, the associations between these embedding vectors and users' feedback are a mixture of the causal part that describes why an item is preferred by a user, and the non-causal part that merely reflects the statistical dependencies between users and items, for example, the exposure mechanism, public opinions, display position, etc. However, existing RSs mostly ignored the striking differences between the causal parts and non-causal parts when using these embedding vectors. In this paper, we propose a model-agnostic framework named IV4Rec that can effectively decompose the embedding vectors into these two parts, hence enhancing recommendation results. Specifically, we jointly consider users' behaviors in search scenarios and recommendation scenarios. Adopting the concepts in causal analysis, we embed users' search behaviors as instrumental variables (IVs), to help decompose original embedding vectors in recommendation, i.e., treatments. IV4Rec then combines the two parts through deep neural networks and uses the combined results for recommendation. IV4Rec is modelagnostic and can be applied to a number of existing RSs such as DIN and NRHUB. Experimental results on both public and proprietary industrial datasets demonstrate that IV4Rec consistently enhances RSs and outperforms a framework that jointly considers search and recommendation. CCS CONCEPTS• Information systems → Recommender systems.

show abstract

Section: Related Workmentioning

confidence: 99%

A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

Si,

Han,

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this work the authors explicitly avoided offline training on operator data, citing the well-known issues of insufficient action coverage. Nevertheless, batch or off-line RL methods (Ernst et al, 2005;Riedmiller, 2005;Lange et al, 2012;Levine et al, 2020) have been successfully used in settings where a fixed policy or value function is extracted from a data-set, with several practical applications (Pietquin et al, 2011;Shortreed et al, 2011;Swaminathan et al, 2017;Levine et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

GVFs in the real world: making predictions online for water treatment

Janjua,

Shah,

White

et al. 2023

Mach Learn

View full text Add to dashboard Cite

In this paper we investigate the use of reinforcement-learning based prediction approaches for a real drinking-water treatment plant. Developing such a prediction system is a critical step on the path to optimizing and automating water treatment. Before that, there are many questions to answer about the predictability of the data, suitable neural network architectures, how to overcome partial observability and more. We first describe this dataset, and highlight challenges with seasonality, nonstationarity, partial observability, and heterogeneity across sensors and operation modes of the plant. We then describe General Value Function (GVF) predictions—discounted cumulative sums of observations–and highlight why they might be preferable to classical n-step predictions common in time series prediction. We discuss how to use offline data to appropriately pre-train our temporal difference learning (TD) agents that learn these GVF predictions, including how to select hyperparameters for online fine-tuning in deployment. We find that the TD-prediction agent obtains an overall lower normalized mean-squared error than the n-step prediction agent. Finally, we show the importance of learning in deployment, by comparing a TD agent trained purely offline with no online updating to a TD agent that learns online. This final result is one of the first to motivate the importance of adapting predictions in real-time, for non-stationary high-volume systems in the real world.

show abstract

“…1 While we focus on online learning environments, there is a rich line of work that studies offline learning of optimal policies after the experimental or observational data is collected (Kitagawa and Tetenov 2018, Kosorok and Laber 2019, Narita et al 2019, Athey and Wager 2021, Kallus and Zhou 2021, Narita et al 2021, Shi et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Self-fulfilling Bandits: Dynamic Selection in Algorithmic Decision-making

Li¹,

Luo²,

Zhang³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we study endogeneity problems in algorithmic decision-making where data and actions are interdependent. When there are endogenous covariates in a contextual multi-armed bandit model, a novel bias (self-fulfilling bias) arises because the endogeneity of the covariates spills over to the actions. We propose a class of algorithms to correct for the bias by incorporating instrumental variables into leading online learning algorithms. These algorithms also attain regret levels that match the best known lower bound for the cases without endogeneity. To establish the theoretical properties, we develop a general technique that untangles the interdependence between data and actions.

show abstract

Debiased Off-Policy Evaluation for Recommendation Systems

Cited by 9 publications

References 8 publications

A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

GVFs in the real world: making predictions online for water treatment

Self-fulfilling Bandits: Dynamic Selection in Algorithmic Decision-making

Contact Info

Product

Resources

About