Ordering the suggestions of a spellchecker without using context

Abstract

Having located a misspelling, a spellchecker generally offers some suggestions for the intended word. Even without using context, a spellchecker can draw on various types of information in ordering its suggestions. A series of experiments is described, beginning with a basic corrector that implements a well known algorithm for reversing single simple errors, and making successive enhancements to take account of substring matches, pronunciation, known error patterns, syllable structure and word frequency. The improvement in the ordering produced by each enhancement is measured on a large corpus of misspellings. The final version is tested on other corpora against a widely used commercial spellchecker and a research prototype.

Managing misspelled queries in IR applications

“…Focusing first on entire dictionary entries, spelling correction is a well known subject matter in NLP (Mitton, 2009;Reynaert, 2004;Savary, 2001;Vilares et al, 2004), often based on the notion of edit distance 2 (Levenshtein, 1966). When dealing with misspelled queries, the aim is to replace the erroneous term or terms in the query with those considered to be the correct ones and whose edit distance with regard to the former is the smallest possible.…”

“…It is interesting to note that practical constraints for solving misspellings in IR systems are different from those present in Text Processing systems. In the latter, the usual solution consists in presenting the user with a set of candidate corrections and lower first-guess accuracy is usually tolerated (Mitton, 2009). However, in IR systems, this kind of interaction is impractical.…”

“…• Isolated-word error correction (Mitton, 2009;Savary, 2001;Vilares et al, 2004), which tries to correct a single query term at a time, limiting the possibility of correction to non-word errors. In this sense, this kind of technique could fail to detect real-word errors, i.e.…”

ETS Lexical Associations System for the COGALEX-4 Shared Task

“…This was improved to 16.1% ( Figure 2B), and with additional filters -to 18.6% (see section 4.2). 5 A list of n-best suggestions is standard approach for presenting candidate corrections for misspellings (Flor, 2013;Mitton, 2008). Also, precision "at n documents" is a well known evaluation approach in information retrieval (Manning et al, 2008).…”

Introduction of the weight edition errors in the Levenshtein distance

“…Ndiaye and Faltin [7] proposed an alternative method of alpha code, who defined a system of suitable spelling correction for learning the French language, based on the method of alpha code modified by combining other techniques such as phonetic reinterpretation, in case where the first method does not find solutions.  A critical analysis of existing systems for spell checker, realized by Souque [8] and Mitton [9], confirms that these systems have limitations in the proposed solutions to some type of erroneous word.…”