Maximizing Cumulative User Engagement in Sequential Recommendation

Abstract

To maximize cumulative user engagement (e.g. cumulative clicks) in sequential recommendation, it is o en needed to tradeo two potentially con icting objectives, that is, pursuing higher immediate user engagement (e.g., click-through rate) and encouraging user browsing (i.e., more items exposured). Existing works o en study these two tasks separately, thus tend to result in sub-optimal results. In this paper, we study this problem from an online optimization perspective, and propose a exible and practical framework to explicitly tradeo longer user browsing length and high immediate user engagement. Speci cally, by considering items as actions, user's requests as states and user leaving as an absorbing state, we formulate each user's behavior as a personalized Markov decision process (MDP), and the problem of maximizing cumulative user engagement is reduced to a stochastic shortest path (SSP) problem. Meanwhile, with immediate user engagement and quit probability estimation, it is shown that the SSP problem can be e ciently solved via dynamic programming. Experiments on real-world datasets demonstrate the e ectiveness of the proposed approach. Moreover, this approach is deployed at a large E-commerce platform, achieved over 7% improvement of cumulative clicks.

Modeling Attrition in Recommender Systems with Departing Bandits

“…More broadly, our RS learning problem falls under the domain of reinforcement learning (RL). Existing RL litera-ture that considers departing users in RSs include Zhao et al (2020b); Lu and Yang (2016); Zhao et al (2020a). While Zhao et al (2020b) handle users of a single type that depart the RS within a bounded number of interactions, our work deals with multiple user types.…”

“…Existing RL litera-ture that considers departing users in RSs include Zhao et al (2020b); Lu and Yang (2016); Zhao et al (2020a). While Zhao et al (2020b) handle users of a single type that depart the RS within a bounded number of interactions, our work deals with multiple user types. In contrast to Zhao et al (2020a), we consider an online setting and provide regret guarantees that do not require bounded horizon.…”

User Response Prediction in Online Advertising

“…Recent studies have modeled user engagement using multi-objective optimizations. So two recommending and online advertising are optimized together to satisfy user experience in the long-term [180,181].…”

ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actor

“…Different from traditional recommender systems which assume that the number of recommended items is fixed, a sequential recommender system keeps recommending items to a user until the user quits the current service/session [34,13]. In sequential recommendation, as depicted in Figure 1, users have the option to browse endless items in one session and can restart a new session after they quit the old one [39]. To this end, an ideal sequential recommender system would be expected to achieve i) low return time between sessions, i.e., high frequency of user visits; and ii) large session length so that more items can be Despite great importance, unfortunately, how to effectively improve long-term engagement in sequential recommendation remains largely uninvestigated.…”

“…However, they are usually based on strong assumptions such as recommendation diversity will increase long-term engagement [32,41]. In fact, the relationship between recommendation diversity and long-term engagement is largely empirical, and how to measure diversity properly is also unclear [39].…”