Aggregating E-commerce Search Results from Heterogeneous Sources via Hierarchical Reinforcement Learning

Takanobu, Ryuichi; Zhuang, Tao; Huang, Minlie; Feng, Jun; Tang, Haitao; Zheng, Bo

doi:10.1145/3308558.3313455

Cited by 15 publications

(7 citation statements)

References 46 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The heterogeneous data sources in online shopping are different product categories such as a shoe brand group or a particular topic group, each of which is a ranking system. A new model in [108] is proposed to decompose the task into two sub-tasks. The first one is to select a data source for the current page of search results based on historical users' clicks on previous pages.…”

Section: Search Results Aggregationmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Cai

Cui

Xiong

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Data processing and analytics are fundamental and pervasive. Algorithms play a vital role in data processing and analytics where many algorithm designs have incorporated heuristics and general rules from human knowledge and experience to improve their effectiveness. Recently, reinforcement learning, deep reinforcement learning (DRL) in particular, is increasingly explored and exploited in many areas because it can learn better strategies in complicated environments it is interacting with than statically designed algorithms. Motivated by this trend, we provide a comprehensive review of recent works focusing on utilizing DRL to improve data processing and analytics. First, we present an introduction to key concepts, theories, and methods in DRL. Next, we discuss DRL deployment on database systems, facilitating data processing and analytics in various aspects, including data organization, scheduling, tuning, and indexing. Then, we survey the application of DRL in data processing and analytics, ranging from data preparation, natural language processing to healthcare, fintech, etc. Finally, we discuss important open challenges and future research directions of using DRL in data processing and analytics.

show abstract

Section: Search Results Aggregationmentioning

confidence: 99%

“…The problem is solved by formulating the sub-task as an RL task to let an agent fill up the sequence. However, one limitation of this method is that lacking full annotations of item relevance scores may constrain the model's performance on various scenarios [108].…”

Section: Search Results Aggregationmentioning

confidence: 99%

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Cai

Cui

Xiong

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…The heterogeneous data sources in online shopping are different product categories such as a shoe brand group or a particular topic group, each of which is a ranking system. A new HRL model in [89] is proposed to decompose the task into two sub-tasks. The first one is to select a data source for the current page of search results based on historical users' clicks on previous pages.…”

Section: Search Results Aggregationmentioning

confidence: 99%

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Cai¹,

Cui²,

Xiong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Data processing and analytics are fundamental and pervasive. Algorithms play a vital role in data processing and analytics where many algorithm designs have incorporated heuristics and general rules from human knowledge and experience to improve their effectiveness. Recently, reinforcement learning, deep reinforcement learning (DRL) in particular, is increasingly explored and exploited in many areas because it can learn better strategies in complicated environments it is interacting with than statically designed algorithms. Motivated by this trend, we provide a comprehensive review of recent works focusing on utilizing deep reinforcement learning to improve data processing and analytics. First, we present an introduction to key concepts, theories, and methods in deep reinforcement learning. Next, we discuss deep reinforcement learning deployment on database systems, facilitating data processing and analytics in various aspects, including data organization, scheduling, tuning, and indexing. Then, we survey the application of deep reinforcement learning in data processing and analytics , ranging from data preparation, natural language interface to healthcare, fintech, etc. Finally, we discuss important open challenges and future research directions of using deep reinforcement learning in data processing and analytics.

show abstract

“…The reason is two-fold. First, the two-stage decomposition is a popular choice to address large, combinatorial action space problems [27,32]. Each subtask is relatively simpler to solve than the original task and the original task can be further solved by combining the results of the subtasks.…”

Section: Introductionmentioning

confidence: 99%

Spending Money Wisely

Sun

Weng

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Online electronic coupon (e-coupon) is becoming a primary tool for e-commerce platforms to attract users to place orders. E-coupons are the digital equivalent of traditional paper coupons which provide customers with discounts or gifts. One of the fundamental problems related is how to deliver e-coupons with minimal cost while users' willingness to place an order is maximized. We call this problem the coupon allocation problem. This is a non-trivial problem since the number of regular users on a mature e-platform often reaches hundreds of millions and the types of e-coupons to be allocated are often multiple. The policy space is extremely large and the online allocation has to satisfy a budget constraint. Besides, one can never observe the responses of one user under different policies which increases the uncertainty of the policy making process. Previous work fails to deal with these challenges. In this paper, we decompose the coupon allocation task into two subtasks: the user intent detection task and the allocation task. Accordingly, we propose a two-stage solution: at the first stage (detection stage), we put forward a novel Instantaneous Intent Detection Network (IIDN) which takes the user-coupon features as input and predicts user real-time intents; at the second stage (allocation stage), we model the allocation problem as a Multiple-Choice Knapsack Problem (MCKP) and provide a computational efficient allocation method using the intents predicted at the detection stage. Long Short Term Memory (LSTM) and a special attention mechanism are applied on IIDN to better describe temporal dependencies of sequential features. And we manage to solve the imbalanced label problem for the user intent detection task with a brand new perspective by using the logical relationship between multiple user intents. We conduct * Both authors contributed equally to this research.

show abstract

Aggregating E-commerce Search Results from Heterogeneous Sources via Hierarchical Reinforcement Learning

Cited by 15 publications

References 46 publications

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Spending Money Wisely

Contact Info

Product

Resources

About