SlateQ: A Tractable Decomposition for Reinforcement Learning with Recommendation Sets

Ie, Eugene; Jain, Vihan; Wang, Jing; Narvekar, Sanmit; Agarwal, Ritesh; Wu, Rui; Cheng, Heng-Tze; Chandra, Tushar; Boutilier, Craig

doi:10.24963/ijcai.2019/360

Cited by 91 publications

(81 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other enhancements include incorporating contextual data [5]. Most recently, Chen et al [10] and Ie et al [23] showed success in applying reinforcement learning techniques in YouTube recommender systems. Our work does not deal with designing a recommender system, nor does it attempt to reverse engineer the YouTube recommender.…”

Section: Recommender Systems and Video Recommendationmentioning

confidence: 99%

“…The first gap measures and estimates the effects of recommender systems in complex social systems. The main goals of recommender systems are maximizing the chance that a user clicks on an item in the next step [4,16,17,48] or in a longer time horizon [5,10,23]. However, recommendation in social systems remains as an open problem for two reasons: (1) a limited conceptual understanding of how finite human attention is allocated over the network of content, in which some items gain popularity at the expense of, or with the assistance of others; (2) the computational challenge of jointly recommending a large collection of items.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimating Attention Flow in Online Video Networks

Rizoiu

Xie

2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

Online videos have shown tremendous increase in Internet traffic. Most video hosting sites implement recommender systems, which connect the videos into a directed network and conceptually act as a source of pathways for users to navigate. At present, little is known about how human attention is allocated over such large-scale networks, and about the impacts of the recommender systems. In this paper, we first construct the Vevo network -a YouTube video network with 60,740 music videos interconnected by the recommendation links, and we collect their associated viewing dynamics. This results in a total of 310 million views every day over a period of 9 weeks. Next, we present large-scale measurements that connect the structure of the recommendation network and the video attention dynamics. We use the bow-tie structure to characterize the Vevo network and we find that its core component (23.1% of the videos), which occupies most of the attention (82.6% of the views), is made out of videos that are mainly recommended among themselves. This is indicative of the links between video recommendation and the inequality of attention allocation. Finally, we address the task of estimating the attention flow in the video recommendation network. We propose a model that accounts for the network effects for predicting video popularity, and we show it consistently outperforms the baselines. This model also identifies a group of artists gaining attention because of the recommendation network. Altogether, our observations and our models provide a new set of tools to better understand the impacts of recommender systems on collective social attention.

show abstract

Section: Recommender Systems and Video Recommendationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Estimating Attention Flow in Online Video Networks

Rizoiu

Xie

2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

show abstract

“…The current trend in this direction is to take into account complex user behaviours and knowledge graph information to achieve high efficiency with a large amount of data and large number of items [151]. The application of reinforcement learning techniques in industrial recommender systems is also prevalent, such as in YouTube [152] and Alibaba [153]. The development of deep reinforcement learning-based recommender systems will continue to be a hot area and will be more heavily driven by real-world industrial applications.…”

Section: Reinforcement Learning In Recommender Systemsmentioning

confidence: 99%

Artificial intelligence in recommender systems

Zhang

Jin

2020

Complex Intell. Syst.

206

View full text Add to dashboard Cite

Recommender systems provide personalized service support to users by learning their previous behaviors and predicting their current preferences for particular products. Artificial intelligence (AI), particularly computational intelligence and machine learning methods and algorithms, has been naturally applied in the development of recommender systems to improve prediction accuracy and solve data sparsity and cold start problems. This position paper systematically discusses the basic methodologies and prevailing techniques in recommender systems and how AI can effectively improve the technological development and application of recommender systems. The paper not only reviews cutting-edge theoretical and practical contributions, but also identifies current research issues and indicates new research directions. It carefully surveys various issues related to recommender systems that use AI, and also reviews the improvements made to these systems through the use of such AI approaches as fuzzy techniques, transfer learning, genetic algorithms, evolutionary algorithms, neural networks and deep learning, and active learning. The observations in this paper will directly support researchers and professionals to better understand current developments and new directions in the field of recommender systems using AI.

show abstract

“…Methods given in [10], [15]- [19], [30] explicitly consider the impacts of the co-displayed items when generating recommendation lists. Some of them [10], [15]- [17] are re-ranking methods.…”

Section: Preliminariesmentioning

confidence: 99%

“…We independently develop a similar idea as [17], where the main difference is that we try to replace the original ranking mechanism in the system with new strategies to generate lists directly as [18]- [20]. The work in [18] is trying to optimize a simplified objective, and the one in [19] makes additional assumptions when solving the list recommendation task. For the method in [20], it is based on conditional variational auto-encoder (CVAE) [21].…”

Section: Introductionmentioning

confidence: 99%

Co-Displayed Items Aware List Recommendation

Song

Zhou

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Existing recommender systems usually generate personalized recommendation lists based on the estimation of the preference scores over user-item pairs, while ignoring the impacts of the entire display list that plays a central part in the decision making process of a user. This leaves us an opportunity to generate better recommendation results by considering the impacts of all offered choices. However, such an extension cannot be handled efficiently by traditional top-k list recommendation methods, due to the entire list dependency issue which means a complete list of items is needed before we can precisely measure any item preference among the list. In this paper, we propose a Co-displayed Items Aware (CDIA) list generation approach, which is based on the reinforcement learning architecture, and can efficiently generate high-utility lists. Specifically, we propose CDIA-Sim to predict users' preferences, which considers the impacts of the co-displayed items. Then, to overcome the entire list dependency issue in the list recommendation task, we utilize the reinforcement learning technique and design CDIA-RL to generate high-utility lists. Experimental results show that CDIA-Sim achieves significant improvements in modeling user-item preferences, and CDIA-RL can yield lists efficiently and effectively, illustrating better performance than other competitors. INDEX TERMS Co-displayed items, list recommendation, reinforcement learning.

show abstract

SlateQ: A Tractable Decomposition for Reinforcement Learning with Recommendation Sets

Cited by 91 publications

References 17 publications

Estimating Attention Flow in Online Video Networks

Estimating Attention Flow in Online Video Networks

Artificial intelligence in recommender systems

Co-Displayed Items Aware List Recommendation

Contact Info

Product

Resources

About