Improving question retrieval in community question answering service using dependency relations and question classification

Abstract

To build an effective community question answering (cQA) service, determining ways to obtain questions similar to an input query question is a significant research issue. The major challenges for question retrieval in cQA are related to solving the lexical gap problem and estimating the relevance between questions. In this study, we first solve the lexical gap problem using a translation-based language model (TRLM). Thereafter, we determine features and methods that are competent for estimating the relevance between two questions. For this purpose, we explore ways to use the results of a dependency parser and question classification for category information. Head-dependent pairs are first extracted as bigram features, called dependency bigrams, from the analysis results of the dependency parser. The probability of each category is estimated using the softmax approach based on the scores of the classification results. Subsequently, we propose two retrieval models-the dependency-based model (DM) and category-based model (CM)-and they are applied to the previous model, TRLM. The experimental results demonstrate that the proposed methods significantly improve the performance of question retrieval in cQA services.

Improving search and recommendation by asking clarifying questions

“…Interactive retrieval methods have also been studied in community-based question answering (cQA). Successful applications in the field include expert finding [216,217], question retrieval [12,30], understanding and summarizing answers [120], question 2. Question-based Document Search routing in providing answers for unanswered questions [112], and inference rules discovery from text [116].…”

“…They highlight a valuable investigation for considering both question contents and the asker's social interactions. Bae and Ko [12] instead presented a translation-based language model to solve the lexical gap problem for retrieving questions. To solve the cold-start problems, Wan et al [188] and Zhao et al [216] exploited knowledge from multiple sources to support question answering.…”

Towards Question-based High-recall Information Retrieval

“…Interactive retrieval methods have also been studied in community-based question answering (cQA). Successful applications in the field include expert finding [Zhao et al 2014[Zhao et al , 2016, question retrieval [Bae and Ko 2019;, understanding and summarizing answers [Liu et al 2008], question routing in providing answers for unanswered questions [Li and King 2010], and inference rules discovery from text [Lin and Pantel 2001]. Zhao et al [2016] proposed a random-walk-based learning method with recurrent neural networks from a novel viewpoint of learning ranking metric embeddings to search the right experts for answering the questions.…”

“…They highlight a valuable investigation for considering both question contents and the asker's social interactions. Bae and Ko [2019] instead presented a translation-based language model to solve the lexical gap problem for retrieving questions. To solve the cold-start problems, Wan et al [2018] and Zhao et al [2014] exploited knowledge from multiple sources to support question answering.…”

Implementation of Question Entailment in Question Answering System for Children’s Health Topic

“…This research focuses on implementing the research conducted by Asma Ben et al [4] by applying the Question Entailment method to the Question Answering System [5], which can return text quotes and even phrases into answers. This method will be assisted by TF-IDF [6] and Bigram [7], which are tested using classification algorithms such as Support Vector Machine (SVM) and the entailment of new questions with dataset questions.…”