Conversational Recommender System

Abstract

A personalized conversational sales agent could have much commercial potential. E-commerce companies such as Amazon, eBay, JD, Alibaba etc. are piloting such kind of agents with their users. However, the research on this topic is very limited and existing solutions are either based on single round adhoc search engine or traditional multi round dialog system. They usually only utilize user inputs in the current session, ignoring users' long term preferences. On the other hand, it is well known that sales conversion rate can be greatly improved based on recommender systems, which learn user preferences based on past purchasing behavior and optimize business oriented metrics such as conversion rate or expected revenue. In this work, we propose to integrate research in dialog systems and recommender systems into a novel and unified deep reinforcement learning framework to build a personalized conversational recommendation agent that optimizes a per session based utility function.In particular, we propose to represent a user conversation history as a semi-structured user query with facet-value pairs. This query is generated and updated by belief tracker that analyzes natural language utterances of user at each step. We propose a set of machine actions tailored for recommendation agents and train a deep policy network to decide which action (i.e. asking for the value of a facet or making a recommendation) the agent should take at each step. We train a personalized recommendation model that uses both the user's past ratings and user query collected in the current conversational session when making rating predictions and generating recommendations. Such a conversational system often tries to collect user preferences by asking questions. Once enough user preference is collected, it makes personalized recommendations to the user. We perform both simulation experiments and real online user studies to demonstrate the effectiveness of the proposed framework.

Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems

“…It is worth noting that besides s his , the other three vectors are all derived from the RC component. We claim that this is a key difference from existing conversational systems [9,23,26,32,40]; i.e., the CC needs to take information from the RC to decide the dialogue action. In contrast to EAR, the recent conversational recommendation method CRM [32] makes decisions based only on the belief tracker that records the preferred attributes of the user, which makes it less informative.…”

“…We claim that this is a key difference from existing conversational systems [9,23,26,32,40]; i.e., the CC needs to take information from the RC to decide the dialogue action. In contrast to EAR, the recent conversational recommendation method CRM [32] makes decisions based only on the belief tracker that records the preferred attributes of the user, which makes it less informative. As such, CRM is less effective especially when the number of attributes is large (their experiments only deal with 5 attributes, which is insufficient for real-world applications).…”

“…This paper considers conversational recommendation as an inherently multi-round scenario, where a CRS interacts with the user by asking attributes and recommending items multiple times until the task succeeds or the user leaves. To distinguish the two, we term the setting single-round where the CRS only makes recommendations once, ending the session regardless of the outcome, as in [9,32]. We now introduce the notation used to formalize our setting.…”

“…We denote the set of all attributes as P and use p to denote a specific attribute. Following [32,40], a CRS session is started with u's specification of a preferred attribute p 0 , then the CRS filters out candidate items that contain the preferred attribute p 0 . Then in each turn t (t = 1, 2, ...,T ; T denotes the last turn of the session), the CRS needs to choose an action: recommend or ask:…”

“…The output layer is normalized to be a probability distribution over all actions by so f tmax. In terms of the action space, we follow the previous method [32], which includes all attributes P and a dedicated action for recommendation. To be specific, we define the action space as A = {a r ec ∪ {a ask (p)|p ∈ P}}, which is of size |P | + 1.…”