Managing misspelled queries in IR applications

Vilares, Jesús; Vilares, Manuel; Otero, Juan María Martínez

doi:10.1016/j.ipm.2010.08.004

Cited by 12 publications

(17 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding to CLIR in particular, we also intend to study the effects of the input parallel corpus on the alignment process with respect to: (a) the minimal input required, following the example of McNamee et al (2009); in the particular case of the n-gram alignment algorithm presented in the this work, the quality of the first phase word-level alignment, that is, if this word alignment can be simplified in order to reduce its associated computational costs. Moreover, we want to take advantage of our experience in the study of the impact of misspellings in monolingual IR systems (Vilares et al, 2011) and to extend that work to the case of CLIR systems.…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, these systems are able to cope not only with spelling errors, but also with OOV words and variants (Vilares et al, 2011;Lee and Ahn, 1996;Mustafa and Al-Radaideh, 2004), in contrast to classical conflation techniques based on stemming, lemmatization or morphological analysis, which are negatively affected by these phenomena.…”

Section: Background and Related Workmentioning

confidence: 99%

“…According to the results of previous related work ( McNamee and Mayfield, 2004a,b;Hollink et al, 2004;Vilares et al, 2011), 4-grams (n-grams of four characters) showed promising, so we decided to use n=4 as n-gram size. No stopword removal was applied at this point.…”

Section: Indexing-retrieval Processesmentioning

confidence: 99%

See 2 more Smart Citations

On the feasibility of character n-grams pseudo-translation for Cross-Language Information Retrieval tasks

Vilares

Alonso

et al. 2016

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

The field of Cross-Language Information Retrieval relates techniques close to both the Machine Translation and Information Retrieval fields, although in a context involving characteristics of its own. The present study looks to widen our knowledge about the effectiveness and applicability to that field of nonclassical translation mechanisms that work at character n-gram level. For the purpose of this study, an n-gram based system of this type has been developed. This system requires only a bilingual machine-readable dictionary of n-grams, automatically generated from parallel corpora, which serves to translate queries previously n-grammed in the source language. n-Gramming is then used as an approximate string matching technique to perform monolingual text retrieval on the set of n-grammed documents in the target language.The tests for this work have been performed on CLEF collections for seven European languages, taking English as the target language. The performance attained, close to the upper baseline, confirms the validity of character n-gram based approaches for Cross Language Information Retrieval tasks, both for indexing-retrieval and translation purposes, these not being tied to a given implementation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the feasibility of character n-grams pseudo-translation for Cross-Language Information Retrieval tasks

Vilares

Alonso

et al. 2016

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

show abstract

“…Having analyzed the state of the art, we consider here two different strategies for dealing with misspelled queries (Manning et al, 2008;Vilares et al, 2011): one that operates at word level and another one that operates at subword level.…”

Section: Processing Misspelled Queriesmentioning

confidence: 99%

“…Our practical experience suggests that the inability to deal with misspelled words is a major source of translation errors for the machine translation engines used in CLIR systems for query translation. In order to do this, we will take advantage of our previous experience both in the study of the impact of misspelled queries on monolingual IR (Vilares et al, 2011) and in character n-gram based CLIR (Vilares et al, 2016). Thus, we will make a comparative analysis of the effectiveness of two possible strategies which are reflected in three specific techniques with different levels of integration of knowledge and linguistic resources.…”

Section: Introductionmentioning

confidence: 99%

Studying the effect and treatment of misspelled queries in Cross-Language Information Retrieval

Vilares

Alonso

Doval

et al. 2016

Information Processing & Management

Self Cite

View full text Add to dashboard Cite

In contrast with their monolingual counterparts, little attention has been paid to the effects that misspelled queries have on the performance of Cross-Language Information Retrieval (CLIR) systems. The present work makes a first attempt to fill this gap by extending our previous work on monolingual retrieval in order to study the impact that the progressive addition of misspellings to input queries has, this time, on the output of CLIR systems. Two approaches for dealing with this problem are analyzed in this paper. Firstly, the use of automatic spelling correction techniques for which, in turn, we consider two algorithms: the first one for the correction of isolated words and the second one for a correction based on the linguistic context of the misspelled word. The second approach to be studied is the use of character n-grams both as index terms and translation units, seeking to take advantage of their inherent robustness and language-independence. All these approaches have been tested on a from-Spanish-to-English CLIR system, that is, Spanish queries on English documents. Real, user-generated spelling errors have been used under a methodology that allows us to study the effectiveness of the different approaches to be tested and their behavior when confronted with different error rates. The results obtained show the great sensitiveness of classic word-based approaches to misspelled queries, although spelling correction techniques can mitigate such negative effects. On the other hand, the use of character n-grams provides great robustness against misspellings.

show abstract

Using character n-grams to match a list of publications to references in bibliographic databases

2016

View full text Add to dashboard Cite

Managing misspelled queries in IR applications

Cited by 12 publications

References 82 publications

On the feasibility of character n-grams pseudo-translation for Cross-Language Information Retrieval tasks

On the feasibility of character n-grams pseudo-translation for Cross-Language Information Retrieval tasks

Studying the effect and treatment of misspelled queries in Cross-Language Information Retrieval

Using character n-grams to match a list of publications to references in bibliographic databases

Contact Info

Product

Resources

About