Exploiting Relevance Feedback in Knowledge Graph Search

Su, Yu; Yang, Shengqi; Sun, Huan; Srivatsa, Mudhakar; Kase, Sue E.; Vanni, Michelle; Yan, Xifeng

doi:10.1145/2783258.2783320

Cited by 32 publications

(18 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning from User Feedback: Improving (i. e., learning) from user feedback is a well-studied problem in, for instance, information retrieval and web-based search [5,27]. Applications in these disciplines include user feedback that is leveraged in order to improve conversational agents [36], assess the relevance of answers in search queries for knowledge graphs [39], or identify incorrect queries to databases [18]. While [34] makes use of shallow binary feedback for machine translation, it does not account for noisy or adversarial feedback.…”

Section: Related Workmentioning

confidence: 99%

Learning from On-Line User Feedback in Neural Question Answering on the Web

Kratzwald

Feuerriegel

2019

The World Wide Web Conference

View full text Add to dashboard Cite

Question answering promises a means of efficiently searching webbased content repositories such as Wikipedia. However, the systems of this type most prevalent today merely conduct their learning once in an offline training phase while, afterwards, all parameters remain static. Thus, the possibility of improvement over time is precluded. As a consequence of this shortcoming, question answering is not currently taking advantage of the wealth of feedback mechanisms that have become prominent on the web (e. g., buttons for liking, voting, or sharing). This is the first work that introduces a question-answering system for (web-based) content repositories with an on-line mechanism for user feedback. Our efforts have resulted in QApedia-a framework for on-line improvement based on shallow user feedback. In detail, we develop a simple feedback mechanism that allows users to express whether a question was answered satisfactorily or whether a different answer is needed. Even for this simple mechanism, the implementation represents a daunting undertaking due to the complex, multi-staged operations that underlie state-of-the-art systems for neural questions answering. Another challenge with regard to web-based use is that feedback is limited (and possibly even noisy), as the true labels remain unknown. We thus address these challenges through a novel combination of neural question answering and a dynamic process based on distant supervision, asynchronous updates, and an automatic validation of feedback credibility in order to mine high-quality training samples from the web for the purpose of achieving continuous improvement over time. Our QApedia framework is the first question-answering system that continuously refines its capabilities by improving its now dynamic content repository and thus the underlying answer extraction. QApedia not only achieves state-of-the-art results over several benchmarking datasets, but we further show that it successfully manages to learn from shallow user feedback, even when the feedback is noisy or adversarial. Altogether, our extensive experimental evaluation, with more than 2,500 hours of computational experiments, demonstrates that a feedback mechanism as simple as a binary vote (which is widespread on the web) can considerably improve performance when combined with an efficient framework for continuous learning. This paper is published under the Creative Commons Attribution 4.0 International (CC-BY 4.0) license. Authors reserve their rights to disseminate the work on their personal and corporate Web sites with the appropriate attribution.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning from On-Line User Feedback in Neural Question Answering on the Web

Kratzwald

Feuerriegel

2019

The World Wide Web Conference

View full text Add to dashboard Cite

show abstract

“…Image [7], [8], [9], [10], [11], [12], [13] Health [2], [1], [14], [15] Music/ Movie [3], [16], [17] Web Search/Social Media [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43] General [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54],…”

Section: Table 3 Distribution Of Fields Area Of Intent Diversity Infmentioning

confidence: 99%

Review of Intent Diversity in Information Retrieval : Approaches, Models and Trends

Mustakim¹,

Wardoyo²,

Mustofa³

2018

JITeCS

View full text Add to dashboard Cite

The fast increasing volume of information databases made some difficulties for a user to find the information that they need. Its important for researchers to find the best method for challenging this problem. user intention detection can be used to increase the relevancies of information delivered from the information retrieval system. This research used a systematic mapping process to identify what area, approaches, and models that mostly used to detect user intention in information retrieval in four years later. the result of this research identified that item-based approach is still the most approach researched by researchers to identify intent diversity in information retrieval. The used of item-based approach still increasing from 2015 until 2017. 34% paper used topic models in their research. It means that Topic models still the necessary models explored by the researchers in this study.

show abstract

“…Only very recently, Su et al [16] proposed exploiting relevance feedback for improving results of searching a knowledge graph, but not for entity search. For entity ranking using text or semi-structured information, relevance feedback has been more popular [17].…”

Section: Previous and Related Workmentioning

confidence: 99%