Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems

Abstract

Recommender systems are embracing conversational technologies to obtain user preferences dynamically, and to overcome inherent limitations of their static models. A successful Conversational Recommender System (CRS) requires proper handling of interactions between conversation and recommendation. We argue that three fundamental problems need to be solved: 1) what questions to ask regarding item attributes, 2) when to recommend items, and 3) how to adapt to the users' online feedback. To the best of our knowledge, there lacks a unified framework that addresses these problems.In this work, we fill this missing interaction framework gap by proposing a new CRS framework named Estimation-Action-Reflection, or EAR, which consists of three stages to better converse with users. (1) Estimation, which builds predictive models to estimate user preference on both items and item attributes; (2) Action, which learns a dialogue policy to determine whether to ask attributes or recommend items, based on Estimation stage and conversation history; and (3) Reflection, which updates the recommender model when a user rejects the recommendations made by the Action stage. We present two conversation scenarios on binary and enumerated questions, and conduct extensive experiments on two datasets from Yelp and LastFM, for each scenario, respectively. Our experiments demonstrate significant improvements over the state-of-the-art method CRM [32], corresponding to fewer conversation turns and a higher level of recommendation hits. CCS CONCEPTS• Information systems → Users and interactive retrieval; Recommender systems; Personalization; • Human-centered computing → Interactive systems and tools.

Interactive Path Reasoning on Graph for Conversational Recommendation

“…Sec 3). We conduct experiments on the Yelp and LastFM datasets, comparing SCPR with state-of-the-art CRS methods [13,24] which also use the information of user, item and attribute but does not use graph. We analyze the properties of each method under different settings, including different types of questions (binary and enumerated) and different granularity of attributes.…”

“…Note that, in this paper, we do not specifically model different semantics of relations, and only care care whether there is an edge between two vertices for simplicity. In addition, the item, attributes and user information and their relations are also used by existing conversational recommendations systems [13,24]. The difference is that, our CPR organizes such three types of information in graph and leverages on the advantages of graph structure to conduct conversational recommendation.…”

“…(1). It is worth mentioning that our design of RL here reflects another major difference with existing conversational recommendation systems EAR [13] and CRM [24]. Although they also learn policy networks with RL, their policy is to decide which attribute to ask, rather than our choice of whether to ask attribute.…”

“…As discussed by [13], it is easier to make successful recommendations when there are fewer candidate items. The reward follows [13], containing five kinds of rewards, namely, (1) r r ec_suc , a strongly positive reward when the recommendation succeeds, (2) r r ec_f ail , a strongly negative reward when the recommendation fails, (3) r ask _suc , a slightly positive reward when the user accepts an asked attribute, (4) r ask _f ail , a negative reward when the user rejects an asked attribute, and (5) r quit , a strongly negative reward if the session reaches the maximum number of turns. The accumulated reward is the weighted sum of these five.…”

“…Therefore, the CRS equally treats all rejected attributes as does not care and only removes the attributes from the candidate set without further actions like removing all items that contain the rejected attributes. In this scenario, Lei et al [13] distills several key research problems, such as: (1) Which items to recommend? (2) Which attribute to ask?…”