Dependence language model for information retrieval

Gao, Jianfeng; Nie, Jian‐Yun; Wu, Guangyuan; Cao, Guihong

doi:10.1145/1008992.1009024

Cited by 186 publications

(125 citation statements)

References 31 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…Despite their many implementations, retrieval experiments with phrases have shown inconsistent results. Recently, a number of retrieval approaches have investigated the effectiveness of language modeling approach in modeling statistical phrases such as n-grams or proximity-based phrases, showing promising results [18], [19].…”

Section: Previous Workmentioning

confidence: 99%

Concept Based Representations as Complement of Bag of Words in Information Retrieval

Carrillo

2010

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. Information Retrieval models, which do not represent texts merely as collections of the words they contain, but rather as collections of the concepts they contain through synonym sets or latent dimensions, are known as Bag-of-Concepts (BoC) representations. In this paper we use random indexing, which uses co-occurrence information among words to generate semantic context vectors and then represent the documents and queries as BoC. In addition, we use a novel representation, Holographic Reduced Representation, previously proposed in cognitive models, which can encode relations between words. We show that these representations can be successfully used in information retrieval, can associate terms, and when they are combined with the traditional vector space model, they improve effectiveness, in terms of mean average precision.

show abstract

Section: Previous Workmentioning

confidence: 99%

Concept Based Representations as Complement of Bag of Words in Information Retrieval

Carrillo

2010

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

“…Several attempts have been made to define IR models capable of capturing term dependencies. Gao et al [4] proposed a dependency model in which term dependency is captured by a biterm model. The final model combines the unigram model and the biterm model.…”

Section: Related Workmentioning

confidence: 99%

“…However, the relationship between them is simply determined by the scale of the overlap, which does not necessarily reflect the true strength of relationship. In general IR, several models have also been proposed to capture dependencies between terms [4,8,9]. Usually, a dependence model is defined in addition to the traditional bag-of-words or word unigram model.…”

Section: Introductionmentioning

confidence: 99%

Modeling Variable Dependencies between Characters in Chinese Information Retrieval

Shi

Nie

2010

Information Retrieval Technology

Self Cite

View full text Add to dashboard Cite

Abstract. Chinese IR can work on words and/or character n-grams. In previous studies, when several types of index are used, independence is usually assumed between them, which obviously is not true in reality. In this paper, we propose a model for Chinese IR that integrates different types of dependency between Chinese characters. The role of a pair of dependent characters in the matching process is variable, depending on the pair's ability to describe the underlying meaning and to retrieve relevant documents. The weight of the pair is learnt using SVM. Our experiments on TREC and NTCIR Chinese collections show that our model can significantly outperform most existing approaches. The results confirm the necessity to integrate dependent pairs of characters in Chinese IR and to use them according to their possible contribution to IR.

show abstract

“…Dependency parsing is used to analyze syntactic structures as dependency relations among words in a sentence. POS tagging and dependency parsing are widely used in text-based technology, including information extraction [1], machine translation [2], opinion mining [3], information retrieval [4], and so on. POS tagging and dependency parsing are typically designed with a pipelined structure that analyzes POS tags before dependency parsing.…”

Section: Introductionmentioning

confidence: 99%

Joint Model of Korean Part-of-Speech Tagging and Dependency Parsing with Partial Tagged Corpus

Park¹,

Seo²

2015

IJKE

View full text Add to dashboard Cite

Abstract-Most recent studies on part-of-speech (POS) tagging and dependency parsing employ a pipelined model design. However, pipelined structures may decrease performance on account of error propagations. Furthermore, syntactic information is required to improve POS tagging performance. In this paper, we propose a joint model of POS tagging and dependency parsing for the Korean language. Our joint model analyzes the maximum score dependency tree with POS tagging using the graph-based CKY parsing method. We present an effective application method for an additional POS tagged corpus and evaluate the method for POS tagging and dependency parsing. The results show that our model improves accuracy by approximately 2.3% and 1.7% more than a pipelined structure using a hidden Markov model and graph-based dependency parsing, respectively.

show abstract

Dependence language model for information retrieval

Cited by 186 publications

References 31 publications

Concept Based Representations as Complement of Bag of Words in Information Retrieval

Concept Based Representations as Complement of Bag of Words in Information Retrieval

Modeling Variable Dependencies between Characters in Chinese Information Retrieval

Joint Model of Korean Part-of-Speech Tagging and Dependency Parsing with Partial Tagged Corpus

Contact Info

Product

Resources

About