Dimension Projection among Languages based on Pseudo-relevant Documents for Query Translation

Dadashkarimi, Javid; Shahshahani, Mahsa S.; Tebbifakhr, Amirhossein; Faili, Heshaam; Shakery, Azadeh

doi:10.48550/arxiv.1605.07844

Cited by 2 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Queries are expanded by the top 50 terms generated by the feedback model [14,6]. We removed Persian stop words from the queries and documents [4,5]. We used STeP1 [13] in our stemming process in Persian.…”

Section: Methodsmentioning

confidence: 99%

“…In highly inflected languages, bilingual dictionaries contain only original forms of the words. Therefore, in dictionary-based CLIR, retrieval systems are obliged either to stem documents and queries, or to leave them intact [8,4,12], or expand the query with inflections. We opted the query expansion approach which is a widely used approach to compensate the shortage of inflections [9,3,5].…”

Section: How To Evaluate the Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

SS4MCT: A Statistical Stemmer for Morphologically Complex Texts

Dadashkarimi

Esfahani

Faili

et al. 2016

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

There have been multiple attempts to resolve various inflection matching problems in information retrieval. Stemming is a common approach to this end. Among many techniques for stemming, statistical stemming has been shown to be effective in a number of languages, particularly highly inflected languages. In this paper we propose a method for finding affixes in different positions of a word. Common statistical techniques heavily rely on string similarity in terms of prefix and suffix matching. Since infixes are common in irregular/informal inflections in morphologically complex texts, it is required to find infixes for stemming. In this paper we propose a method whose aim is to find statistical inflectional rules based on minimum edit distance table of word pairs and the likelihoods of the rules in a language. These rules are used to statistically stem words and can be used in different text mining tasks. Experimental results on CLEF 2008 and CLEF 2009 English-Persian CLIR tasks indicate that the proposed method significantly outperforms all the baselines in terms of MAP.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: How To Evaluate the Algorithmmentioning

confidence: 99%

SS4MCT: A Statistical Stemmer for Morphologically Complex Texts

Dadashkarimi

Esfahani

Faili

et al. 2016

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The obtained models are used for query prediction and advertisement [10]. A number of works investigate on using the embedded vectors in cross-lingual environments [6,29,2]. [6] employed an offline projection algorithm to bridge the gap between the languages.…”

Section: Low-dimensional Vectorsmentioning

confidence: 99%

“…A number of works investigate on using the embedded vectors in cross-lingual environments [6,29,2]. [6] employed an offline projection algorithm to bridge the gap between the languages. The authors incorporated the vector similarities for building a query language model.…”

Section: Low-dimensional Vectorsmentioning

confidence: 99%

“…Word embedding has limited applications in ad-hoc retrieval but, it is likely to have a burgeon in this area during the next years [21]. However, it has been shown that query-document vector similarity, in which there is not document length normalization nor term frequency consideration, degrades the retrieval performance and thus it is recommend to use these vectors within state-of-theart retrieval frameworks [33,29,6]. Estimating robust language models for documents and queries is a required task in this area.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Low-dimensional Query Projection based on Divergence Minimization Feedback Model for Ad-hoc Retrieval

Dadashkarimi¹,

Sabet²,

Faili³

et al. 2016

Preprint

Self Cite

View full text Add to dashboard Cite

Low-dimensional word vectors have long been used in a wide range of applications in natural language processing. In this paper we shed light on estimating query vectors in ad-hoc retrieval where a limited information is available in the original query. Pseudorelevance feedback (PRF) is a well-known technique for updating query language models and expanding the queries with a number of relevant terms. We formulate the query updating in lowdimensional spaces first with rotating the query vector and then with scaling. These consequential steps are embedded in a queryspecific projection matrix capturing both angle and scaling. In this paper we propose a new but not the most effective technique necessarily for PRF in language modeling, based on the query projection algorithm. We learn an embedded coefficient matrix for each query, whose aim is to improve the vector representation of the query by transforming it to a more reliable space, and then update the query language model. The proposed embedded coefficient divergence minimization model (ECDMM) takes top-ranked documents retrieved by the query and obtains a couple of positive and negative sample sets; these samples are used for learning the coefficient matrix which will be used for projecting the query vector and updating the query language model using a softmax function. Experimental results on several TREC and CLEF data sets in several languages demonstrate effectiveness of ECDMM. The experimental results reveal that the new formulation for the query works as well as state-of-the-art PRF techniques and outperforms state-of-the-art PRF techniques in a TREC collection in terms of MAP,P@5, and P@10 significantly.

show abstract

Dimension Projection among Languages based on Pseudo-relevant Documents for Query Translation

Cited by 2 publications

References 0 publications

SS4MCT: A Statistical Stemmer for Morphologically Complex Texts

SS4MCT: A Statistical Stemmer for Morphologically Complex Texts

Low-dimensional Query Projection based on Divergence Minimization Feedback Model for Ad-hoc Retrieval

Contact Info

Product

Resources

About