A retrospective study of probabilistic context-based retrieval

Wu, H. C.; Luk, Robert W. P.; Wong, Kam-Fai; Kwok, K. L.; Li, Wenjie

doi:10.1145/1076034.1076179

Cited by 5 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The local relevance at a certain location is thought to depend on the document context at that location. This is supported by encouraging results in recent studies by Wu et al [2007Wu et al [ , 2006Wu et al [ , 2005, as well as by Pickens and MacFarlane [2006] using document-contextbased models. By shrinking the context size to unity, we derive the well-known TF-IDF term weights after making some further simplifying assumptions that are similar to the derivations in the language model [Ponte and Croft 1998], the binary independence model [Robertson and Spärck Jones 1976], and the logistic regression model [Cooper et al 1993[Cooper et al , 1992.…”

Section: Related Worksupporting

confidence: 71%

“…It generalizes recent work by Wu et al [2007Wu et al [ , 2006Wu et al [ , 2005. Their work modeled the set of local relevance, {R d,k,q }, as local decision preferences that are defined as the normalized log-odds [Robertson and Spärck Jones 1976] of the local relevance of the corresponding document contexts {c(d, k, n)}.…”

Section: General Modelmentioning

confidence: 85%

“…Wu et al [2007] also showed their retrieval model to obtain high retrieval effectiveness in retrospective experiments, using a fixed-size context that is much shorter than the document length. Their work [Wu et al 2006[Wu et al , 2005 in relevance feedback using a document context model for reranking documents also reported effective retrieval. Their work directly supports the use of document contexts for local relevance decision-making.…”

Section: General Modelmentioning

confidence: 99%

See 2 more Smart Citations

Interpreting TF-IDF term weights as making relevance decisions

Luk

Wong

et al. 2008

ACM Trans. Inf. Syst.

635

295

View full text Add to dashboard Cite

A novel probabilistic retrieval model is presented. It forms a basis to interpret the TF-IDF term weights as making relevance decisions. It simulates the local relevance decision-making for every location of a document, and combines all of these "local" relevance decisions as the "documentwide" relevance decision for the document. The significance of interpreting TF-IDF in this way is the potential to: (1) establish a unifying perspective about information retrieval as relevance decision-making; and (2) develop advanced TF-IDF-related term weights for future elaborate retrieval models. Our novel retrieval model is simplified to a basic ranking formula that directly corresponds to the TF-IDF term weights. In general, we show that the term-frequency factor of the ranking formula can be rendered into different term-frequency factors of existing retrieval systems. In the basic ranking formula, the remaining quantity − log p(r|t ∈ d ) is interpreted as the probability of randomly picking a nonrelevant usage (denoted byr) of term t. Mathematically, we show that this quantity can be approximated by the inverse document-frequency (IDF). Empirically, we show that this quantity is related to IDF, using four reference TREC ad hoc retrieval data collections.

show abstract

Section: Related Worksupporting

confidence: 71%

Section: General Modelmentioning

confidence: 85%

Section: General Modelmentioning

confidence: 99%

See 1 more Smart Citation

Interpreting TF-IDF term weights as making relevance decisions

Luk

Wong

et al. 2008

ACM Trans. Inf. Syst.

635

295

View full text Add to dashboard Cite

show abstract

“…It should not be too small to exclude relevant information or too large to include irrelevant information. So, we compare our RF results with those in the retrospective (RE) experiments by Wu et al [3] where the retrieval model used full relevance judgments. For efficiency, the context size is set to 71.…”

Section: Methodsmentioning

confidence: 99%

Probabilistic document-context based relevance feedback with limited relevance judgments

Luk

Wong

et al. 2006

Proceedings of the 15th ACM International Conference on Information and Knowledge Management - CIKM '06

Self Cite

View full text Add to dashboard Cite

This paper presents our novel relevance feedback (RF) algorithm that uses the probabilistic document-context based retrieval model with limited relevance judgments for document re-ranking. Probabilities of the document-context based retrieval model are estimated from the top N (=20) documents in the initial retrieval. We use document-context based cosine similarity measure to find similar data for better probability estimation in order to reduce the data scarcity problem and the negative weighting problem. Our RF algorithm is promising because its mean average precision is statistically significantly better than the baseline using TREC-6 and TREC-7 data collections.

show abstract

“…Wu et al [5] advocated a document-context based probabilistic retrieval model where scores are obtained using documentcontexts centered on query terms. We develop our documentcontext similarity as follows.…”

Section: Context-based Clusteringmentioning

confidence: 99%

Query-specific clustering of search results based on document-context similarity scores

Dang¹,

Luk²,

Lee

et al. 2006

Proceedings of the 15th ACM International Conference on Information and Knowledge Management - CIKM '06

Self Cite

View full text Add to dashboard Cite

Query difficulty prediction aims to estimate, in advance, whether the answers returned by search engines in response to a query are likely to be useful. This paper proposes new predictors based upon the similarity between the query and answer documents, as calculated by the three different models. It examined the use of anchor text-based document surrogates, and how their similarity to queries can be used to estimate query difficulty. It evaluated the performance of the predictors based on 1) the correlation between the average precision (AP), 2) the precision at 10 (P@10) of the full text retrieved results, 3) a similarity score of anchor text, and 4) a similarity score of full-text, using the WT10g data collection of web data. Experimental evaluation of our research shows that five of our proposed predictors demonstrate reliable and consistent performance across a variety of different retrieval models.

show abstract

A retrospective study of probabilistic context-based retrieval

Cited by 5 publications

References 9 publications

Interpreting TF-IDF term weights as making relevance decisions

Interpreting TF-IDF term weights as making relevance decisions

Probabilistic document-context based relevance feedback with limited relevance judgments

Query-specific clustering of search results based on document-context similarity scores

Contact Info

Product

Resources

About