Axiomatic Analysis of Translation Language Model for Information Retrieval

Karimzadehgan, Maryam; Zhai, ChengXiang

doi:10.1007/978-3-642-28997-2_23

Cited by 29 publications

(21 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of AP and WSJ is in line with previous work on translation language models [16,17], although we consider more documents (AP88-89) and more topics (TREC 1, 2 and 3). The motivation for the use of AP88-89 and WSJ87-92 was that manual runs were included to form the pools of documents for assessment.…”

Section: Data Setsmentioning

confidence: 98%

“…As alternative to the synthetic queries process, Karimzadehgan and Zhai have proposed a family of approaches to estimate pt(w|u) based on mutual information [16,17]. In statistics, mutual information measures the mutual dependence between two random variables by determining how similar the joint distribution p(X, Y ) is to the products of the marginals, p(X)p(Y ).…”

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

“…Mutual information does not guarantee that the self-translation probability p(w|w) (i.e., the probability of translating a word w to itself) is higher than any other translation probability [16] (it is possible that p(w|u) > p(w|w)). An axiomatic analysis of translation language models has shown that this is not a desired situation because documents that match a query word qi exactly may receive a lower score than documents that match translations of qi that have a translation probability higher than the self-translation probability [17]. To overcome this issue, a heuristic has been proposed to control the effect of self-translation via a parameter α:…”

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

“…Similarly, an alternative heuristic is to impose constant self-translation probabilities for all words in the vocabulary [17], i.e., setting p(u|u) to a constant value s for every u. This produces another variant of TLM-MI, which we refer to as TLM-MI-s, where pt(w|u) is estimated according to:…”

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

“…The Dirichlet language model constituted the baseline method (LM), in line with the previous experiments on translation language models of Karimzadehgan and Zhai [16,17]. We mimic their experimental methodology: parameters were tuned per-collection and per-topic set (but not on a perquery basis) with MAP as objective measure 5 .…”

Section: Baseline -Dirichlet Language Modelmentioning

confidence: 99%

See 4 more Smart Citations

Integrating and Evaluating Neural Word Embeddings in Information Retrieval

Zuccon

Koopman

Bruza

et al. 2015

Proceedings of the 20th Australasian Document Computing Symposium

View full text Add to dashboard Cite

Recent advances in neural language models have contributed new methods for learning distributed vector representations of words (also called word embeddings). Two such methods are the continuous bag-of-words model and the skipgram model. These methods have been shown to produce embeddings that capture higher order relationships between words that are highly effective in natural language processing tasks involving the use of word similarity and word analogy. Despite these promising results, there has been little analysis of the use of these word embeddings for retrieval.Motivated by these observations, in this paper, we set out to determine how these word embeddings can be used within a retrieval model and what the benefit might be. To this aim, we use neural word embeddings within the well known translation language model for information retrieval. This language model captures implicit semantic relations between the words in queries and those in relevant documents, thus producing more accurate estimations of document relevance.The word embeddings used to estimate neural language models produce translations that differ from previous translation language model approaches; differences that deliver improvements in retrieval effectiveness. The models are robust to choices made in building word embeddings and, even more so, our results show that embeddings do not even need to be produced from the same corpus being used for retrieval.

show abstract

Section: Data Setsmentioning

confidence: 98%

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

Section: Estimation Of Translation Probability Based On Mutual Informmentioning

confidence: 99%

Section: Baseline -Dirichlet Language Modelmentioning

confidence: 99%

See 3 more Smart Citations

Integrating and Evaluating Neural Word Embeddings in Information Retrieval

Zuccon

Koopman

Bruza

et al. 2015

Proceedings of the 20th Australasian Document Computing Symposium

View full text Add to dashboard Cite

show abstract

An Axiomatic Approach to Diagnosing Neural IR Models

Rennings

Moraes

Hauff

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Traditional retrieval models such as BM25 or language models have been engineered based on search heuristics that later have been formalized into axioms. The axiomatic approach to information retrieval (IR) has shown that the effectiveness of a retrieval method is connected to its fulfillment of axioms. This approach enabled researchers to identify shortcomings in existing approaches and "fix" them. With the new wave of neural net based approaches to IR, a theoretical analysis of those retrieval models is no longer feasible, as they potentially contain millions of parameters. In this paper, we propose a pipeline to create diagnostic datasets for IR, each engineered to fulfill one axiom. We execute our pipeline on the recently released large-scale question answering dataset WikiPassageQA (which contains over 4000 topics) and create diagnostic datasets for four axioms. We empirically validate to what extent wellknown deep IR models are able to realize the axiomatic pattern underlying the datasets. Our evaluation shows that there is indeed a positive relation between the performance of neural approaches on diagnostic datasets and their retrieval effectiveness. Based on these findings, we argue that diagnostic datasets grounded in axioms are a good approach to diagnosing neural IR models.

show abstract

An Intrinsic Framework of Information Retrieval Evaluation Measures

Giner

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Axiomatic Analysis of Translation Language Model for Information Retrieval

Cited by 29 publications

References 9 publications

Integrating and Evaluating Neural Word Embeddings in Information Retrieval

Integrating and Evaluating Neural Word Embeddings in Information Retrieval

An Axiomatic Approach to Diagnosing Neural IR Models

An Intrinsic Framework of Information Retrieval Evaluation Measures

Contact Info

Product

Resources

About