Towards retrieval-based conversational recommendation

Manzoor, Ahtsham; Jannach, Dietmar

doi:10.1016/j.is.2022.102083

Cited by 14 publications

(7 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In case of ambiguity or obscurity, they were allowed to access online portals, e.g., IMDb. Note that regarding the genres, a set of 27 keywords already curated in the context of developing the CRB-CRS [33] was provided to the annotators for their understanding. After briefing, the dataset was splitted evenly for both annotators.…”

Section: Data Annotation Methodologymentioning

confidence: 99%

“…One main advantage of retrieval-based approaches is that the retrieved responses are originally made by humans and thus, are grammatically correct and in themselves semantically meaningful [32]. To this end, both RB-CRS [16] and CRB-CRS [33] were developed in the context of the ReDial dataset [2], and are also included in our evaluation.…”

Section: Totalmentioning

confidence: 99%

“…To this end, case studies with human judges generally investigate questions related to the perceived quality of a system. Common quality aspects, e.g., to evaluate the meaningfulness or consistency of system-responses are investigated in recent work on CRS, see, e.g., , [11,24,33]. Therefore, current works on CRS usually rely on a combined approach in their evaluations using both offline metrics and case studies with humans, see e.g., [7,11,12].…”

Section: Crs Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

Manzoor¹,

Jannach²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Conversational recommender systems (CRS) that interact with users in natural language utilize recommendation dialogs collected with the help of paired humans, where one plays the role of a seeker and the other as a recommender. These recommendation dialogs include items and entities to disclose seekers' preferences in natural language. However, in order to precisely model the seekers' preferences and respond consistently, mainly CRS rely on explicitly annotated items and entities that appear in the dialog, and usually leverage the domain knowledge. In this work, we investigate INSPIRED, a dataset consisting of recommendation dialogs for the sociable conversational recommendation, where items and entities were explicitly annotated using automatic keyword or pattern matching techniques. To this end, we found a large number of cases where items and entities were either wrongly annotated or missing annotations at all. The question however remains to what extent automatic techniques for annotations are effective. Moreover, it is unclear what is the relative impact of poor and improved annotations on the overall effectiveness of a CRS in terms of the consistency and quality of responses. In this regard, first, we manually fixed the annotations and removed the noise in the INSPIRED dataset. Second, we evaluate the performance of several benchmark CRS using both versions of the dataset. Our analyses suggest that with the improved version of the dataset, i.e., INSPIRED2, various benchmark CRS outperformed and that dialogs are rich in knowledge concepts compared to when the original version is used. We release our improved dataset (INSPIRED2) publicly at https://github.com/ahtsham58/INSPIRED2.

show abstract

Section: Data Annotation Methodologymentioning

confidence: 99%

Section: Totalmentioning

confidence: 99%

Section: Crs Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

Manzoor¹,

Jannach²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Table 7 provides an overview of the feature requirements considered by the case study participants [140,141] during their decision-making process for developing their conversational recommender systems. The selected feature requirements were instrumental in guiding the participants' model selection.…”

Section: Evaluation Of Findings: Case Studiesmentioning

confidence: 99%

Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review

Farshidi,

Rezaee,

Mazaheri

et al. 2023

Preprint

View full text Add to dashboard Cite

Context: User intent modeling is a crucial process in Natural Language Processing that aims to identify the underlying purpose behind a user’s request, enabling personalized responses. With a vast array of approaches introduced in the literature (over 13,000 papers in the last decade), understanding the related concepts and commonly used models in AI-based systems is essential. Method: We conducted a systematic literature review to gather data on models typically employed in designing conversational recommender systems. From the collected data, we developed a decision model to assist researchers in selecting the most suitable models for their systems. Additionally, we performed two case studies to evaluate the effectiveness of our proposed decision model. Results: Our study analyzed 59 distinct models and identified 74 commonly used features. We provided insights into potential model combinations, trends in model selection, quality concerns, evaluation measures, and frequently used datasets for training and evaluating these models. Contribution: Our study contributes practical insights and a comprehensive understanding of user intent modeling, empowering the development of more effective and personalized conversational recommender systems. With the Conversational Recommender System, researchers can perform a more systematic and efficient assessment of fitting intent modeling frameworks.

show abstract

“…Traditional retrieval methods [23,11] mainly depend on lexical features for the retrieval task which limits them to capture the semantics of the query. In order to address the challenge of lexical gap between user queries and answers, there is a body of work [3,18,29] that trains a semantic retrieval model from labeled data in the form of user queries and matching responses. In recent years, there has been an increasing research [7,13] on unsupervised learning techniques for text-encoder training eliminating the need for annotated data.…”

Section: Related Workmentioning

confidence: 99%

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

Banerjee¹,

Jain²,

Kulkarni³

2023

Preprint

View full text Add to dashboard Cite

In the domain of question-answering in NLP, the retrieval of Frequently Asked Questions (FAQ) is an important sub-area which is well researched and has been worked upon for many languages. Here, in response to a user query, a retrieval system typically returns the relevant FAQs from a knowledge-base. The efficacy of such a system depends on its ability to establish semantic match between the query and the FAQs in real-time. The task becomes challenging due to the inherent lexical gap between queries and FAQs, lack of sufficient context in FAQ titles, scarcity of labeled data and high retrieval latency. In this work, we propose a bi-encoder-based query-FAQ matching model that leverages multiple combinations of FAQ fields (like, question, answer, and category) both during model training and inference. Our proposed Multi-Field Bi-Encoder (MFBE) model benefits from the additional context resulting from multiple FAQ fields and performs well even with minimal labeled data. We empirically support this claim through experiments on proprietary as well as open-source public datasets in both unsupervised and supervised settings. Our model achieves around 27% and 20% better top-1 accuracy for the FAQ retrieval task on internal and open datasets, respectively over the best performing baseline.

show abstract

Towards retrieval-based conversational recommendation

Cited by 14 publications

References 54 publications

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

Contact Info

Product

Resources

About