Improving retrieval accuracy of difficult queries through generalizing negative document language models

Karimzadehgan, Maryam; Zhai, ChengXiang

doi:10.1145/2063576.2063586

Cited by 16 publications

(16 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Information retrieval systems still perform badly for some difficult queries [5,6,7,20]. When a query is difficult, users can reformulate the query or look for articles further down the list to give positive relevance feedback (PRF) to try to improve the search results [9,15,16], or users could give negative relevance feedback (NRF), if such possibility is available, typically to the documents that are wrongly ranked highly. Previous studies on NRF have shown that it is generally not as useful as PRF [10], so most work on relevance feedback is on PRF only.…”

Section: Related Workmentioning

confidence: 99%

“…Previous studies have shown with simulations and often artificial queries that negative relevance feedback (NRF) on documents, in the form of binary feedback, can help to improve the search results efficiently when the query is difficult [15,16,19,29,30]. We continue research on NRF using real search tasks with continuous-valued feedback on document keywords.…”

Section: Introductionmentioning

confidence: 96%

“…We continue research on NRF using real search tasks with continuous-valued feedback on document keywords. Furthermore, studies [16,21] showed that NRF can be made more effective by generalization and aiming for a generalization that is close to the relevant documents. Our system generalizes the negative relevance feedback using documents that are estimated to be relevant by the system and shares these keywords with the user through the visualisation allowing the user to correct the model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling

Peltonen

Strahl

Floréen

2017

Proceedings of the 22nd International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

In difficult information seeking tasks, the majority of topranked documents for an initial query may be non-relevant, and negative relevance feedback may then help find relevant documents. Traditional negative relevance feedback has been studied on document results; we introduce a system and interface for negative feedback in a novel exploratory search setting, where continuous-valued feedback is directly given to keyword features of an inferred probabilistic user intent model. The introduced system allows both positive and negative feedback directly on an interactive visual interface, by letting the user manipulate keywords on an optimized visualization of modeled user intent. Feedback on the interactive intent model lets the user direct the search: Relevance of keywords is estimated from feedback by Bayesian inference, influence of feedback is increased by a novel propagation step, documents are retrieved by likelihoods of relevant versus non-relevant intents, and the most relevant keywords (having the highest upper confidence bounds of relevance) and the most non-relevant ones (having the smallest lower confidence bounds of relevance) are shown as options for further feedback. We carry out task-based information seeking experiments with real users on difficult real tasks; we compare the system to the nearest state of the art baseline allowing positive feedback only, and show negative feedback significantly improves the quality of retrieved information and user satisfaction for difficult tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling

Peltonen

Strahl

Floréen

2017

Proceedings of the 22nd International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

show abstract

“…the relevant documents ]. In [Karimzadehgan and Zhai 2011], a generalized negative language model is developed to penalize/prune the non-relevant documents that are close to the negative language model.…”

Section: Related Workmentioning

confidence: 99%

A Distribution Separation Method Using Irrelevance Feedback Data for Information Retrieval

Zhang

Qian

Hou

et al. 2017

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

In many research and application areas, such as information retrieval and machine learning, we often encounter dealing with a probability distribution which is mixed by one distribution that is relevant to our task in hand and the other that is irrelevant and we want to get rid of. Thus, it is an essential problem to separate the irrelevant distribution from the mixture distribution. This paper is focused on the application in Information Retrieval, where relevance feedback is a widely used technique to build a refined query model based on a set of feedback documents. However, in practice, the relevance feedback set, even provided by users explicitly or implicitly, is often a mixture of relevant and irrelevant documents. Consequently, the resultant query model (typically a term distribution) is often a mixture rather than a true relevance term distribution, leading to a negative impact on the retrieval performance. To tackle this problem, we recently proposed a Distribution Separation Method (DSM), which aims to approximate the true relevance distribution by separating a seed irrelevance distribution from the mixture one. While it achieved a promising performance in an empirical evaluation with simulated explicit irrelevance feedback data, it has not been deployed in the scenario where one should automatically obtain the irrelevance feedback data. In this article, we propose a substantial extension of the basic DSM from two perspectives: developing a further regularization framework and deploying DSM in the automatic irrelevance feedback scenario. Specifically, in order to avoid the output distribution of DSM drifting away from the true relevance distribution when the quality of seed irrelevant distribution (as the input to DSM) is not guaranteed, we propose a DSM regularization framework to constrain the estimation for the relevance distribution. This regularization framework includes three algorithms, each corresponding to a regularization strategy incorporated in the objective function of DSM. In addition, we exploit DSM in automatic (i.e., pseudo) irrelevance feedback, by automatically detecting the seed irrelevant documents via three different document re-ranking methods. We have carried out extensive experiments based on various TREC data sets, in order to systematically evaluate the proposed methods. The experimental results demonstrate the effectiveness of our proposed approaches in comparison with various strong baselines.

show abstract

“…4 It is worth noting at this point that we could have potentially used information induced from the non-relevant documents (F (L f use ) \ Rq) to also improve the query model, Mq;R q , which is used in PoolRank. However, utilizing negative feedback to improve retrieval performance has long been known as an extremely hard task [17,29,37] with the potential merits confined to very difficult queries [37,18].…”

Section: Estimating List Effectivenessmentioning

confidence: 99%

Utilizing relevance feedback in fusion-based retrieval

Rabinovich

Rom²,

Kurland

2014

Proceedings of the 37th International ACM SIGIR Conference on Research &Amp; Development in Information Retrieval

View full text Add to dashboard Cite

Work on using relevance feedback for retrieval has focused on the single retrieved list setting. That is, an initial document list is retrieved in response to the query and feedback for the most highly ranked documents is used to perform a second search. We address a setting wherein the list for which feedback is provided results from fusing several intermediate retrieved lists. Accordingly, we devise methods that utilize the feedback while exploiting the special characteristics of the fusion setting. Specifically, the feedback serves two different, yet complementary, purposes. The first is to directly rank the pool of documents in the intermediate lists. The second is to estimate the effectiveness of the intermediate lists for improved re-fusion. In addition, we present a meta fusion method that uses the feedback for these two purposes simultaneously. Empirical evaluation demonstrates the merits of our approach. As a case in point, the retrieval performance is substantially better than that of using the relevance feedback as in the single list setting. The performance also substantially transcends that of a previously proposed approach to utilizing relevance feedback in fusion-based retrieval.

show abstract

Improving retrieval accuracy of difficult queries through generalizing negative document language models

Cited by 16 publications

References 37 publications

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling

A Distribution Separation Method Using Irrelevance Feedback Data for Information Retrieval

Utilizing relevance feedback in fusion-based retrieval

Contact Info

Product

Resources

About