Estimation of statistical translation models based on mutual information for ad hoc information retrieval

Abstract

As a principled approach to capturing semantic relations of words in information retrieval, statistical translation models have been shown to outperform simple document language models which rely on exact matching of words in the query and documents. A main challenge in applying translation models to ad hoc information retrieval is to estimate a translation model without training data. Existing work has relied on training on synthetic queries generated based on a document collection. However, this method is computationally expensive and does not have a good coverage of query words. In this paper, we propose an alternative way to estimate a translation model based on normalized mutual information between words, which is less computationally expensive and has better coverage of query words than the synthetic query method of estimation. We also propose to regularize estimated translation probabilities to ensure sufficient probability mass for self-translation. Experiment results show that the proposed mutual information-based estimation method is not only more efficient, but also more effective than the synthetic query-based method, and it can be combined with pseudo-relevance feedback to further improve retrieval accuracy. The results also show that the proposed regularization strategy is effective and can improve retrieval accuracy for both synthetic query-based estimation and mutual information-based estimation.

Position-Aligned Translation Model for Citation Recommendation

“…Metzler et al [9] used translation models as a similarity measure for the information flow tracking task. More recently, [2] trained translation model with mutual information of the cooccurent terms, and used it for improving adhoc retrieval. Gao et al [8] extended the translation model to translate between phrases, and used it bridge the vocabulary gap between the Web search query and the page title.…”

“…It has been found that the trained translation model usually underestimates the self-translation probability (translating one term to itself), so it usually boosts the self-translation in retrieval [2]. The ranking function can be formulated as:…”

“…Many efforts have been devoted to address this problem by mining relationships between the terms from the document collection [1,2]. The mined relationships can be used to expand the query by adding related terms.…”

“…Despite the fact that the query and the title is not parallel in the sense of translation as in machine translation (MT), the term relations extracted are shown to be highly useful in IR. Similar approaches have been used successfully in several IR applications such as cross-linguistic retrieval [4,5], question answering [6,7], ad-hoc and Web retrieval [8,2], information flow tracking [9] and citation recommending [10]. In the latter applications, even noisier training data are used such as pairs of queries and relevant documents.…”

A Term Association Translation Model for Naive Bayes Text Classification

“…The term translation probability P t (w i |w) is dierent from the bigram probability P (w i |w) in that the words w i and w are not limited to occur in order and adjacently in the former. Then, the term association information can be integrated into the unigram class model as follows, P (w i |c) = w∈c P t (w i |w)P (w|c), (11) where P (w|c) reects the distribution of words in the training documents of class c, which can be computed via the maximum likelihood estimate. By replacing P (w i |c) in (10) with the one computed by (11), we have…”