A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal errors in words

Ullmann, J. R.

doi:10.1093/comjnl/20.2.141

Cited by 106 publications

(47 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As explained in Section I, a context bound transformation can be regarded as a self-index representation of a kgram index [6], [7], [8], [9]. A k-gram index is a popular alternative for constructing inverted indexes on languages that are not amenable to term tokenization and stemming, and a core component in the highly successful BLAST application for searching in genomic data [12].…”

Section: Application: a K-gram Self-indexmentioning

confidence: 99%

See 1 more Smart Citation

Backwards Search in Context Bound Text Transformations

Petri

Navarro

Culpepper

et al. 2011

2011 First International Conference on Data Compression, Communications and Processing

View full text Add to dashboard Cite

Abstract-The Burrows-Wheeler Transform (BWT) is the basis for many of the most effective compression and selfindexing methods used today. A key to the versatility of the BWT is the ability to search for patterns directly in the transformed text. A backwards search for a pattern P can be performed on a transformed text by iteratively determining the range of suffixes that match P . The search can be further enhanced by constructing a wavelet tree over the output of the BWT in order to emulate a suffix array. In this paper, we investigate new algorithms for search derived from a variation of the BWT whereby rotations are only sorted to a depth k, commonly referred to as a context bound transform. Interestingly, this BWT variant can be used to mimic a k-gram index, which are used in a variety of applications that need to efficiently return occurrences in text position order. In this paper, we present the first backwards search algorithms on the k-BWT, and show how to construct a self-index containing many of the attractive properties of a k-gram index.

show abstract

Section: Application: a K-gram Self-indexmentioning

confidence: 99%

“…Prior to the discovery of the searching capabilities of the BWT, a folklore method for reducing space called a k-gram index was commonly used for substring searches [6], [7], [8], [9]. A k-gram index records the occurrences of each distinct substring of length k in an attempt to mimic the efficiency of inverted indexes.…”

Section: Introductionmentioning

confidence: 99%

Backwards Search in Context Bound Text Transformations

Petri

Navarro

Culpepper

et al. 2011

2011 First International Conference on Data Compression, Communications and Processing

View full text Add to dashboard Cite

show abstract

“…It is a classical AI technique used mostly for error corrections (Riseman&Hanson, 1974;Ullmann, 1977). Finally, a very human-like fuzzy evaluation of strings not contained in the dictionary appeared, classifying them in several classes of peculiarity.…”

Section: Hascheck -A Learning (Semi)automatonmentioning

confidence: 99%

Developing a Spell Checker as an Expert System

Dembitz¹,

Knežević²,

Sokele³

2003

CIT

View full text Add to dashboard Cite

In this paper we discuss some basic concepts of knowledge engineering, expressing our feeling that there is something missing. In our opinion, the missing link, able to clarify all the basic concepts, should be the concept of energy, not in use in knowledge engineering practice and theory at all. Since energy is an extremely abstract concept, which led to misunderstandings even in physics, we humbly introduce it by describing an existing expert system, called Hascheck. Hascheck is a spell checker for Croatian language, implemented as learning (semi)automaton, accessible via E-mail and Web, and being in public use nearly 10 years. We also present here a mathematical model of Hascheck's learning. The model led to the so-called cognoelectrical analogy, which allowed some energy considerations about learning, and knowledge.

show abstract

“…Many studies focus on automatic spelling correction (Accomazzi, Eichhorn, Kurtz, Grant, & Murray, 2000;Angell, Freund, & Willett, 1983;Blair, 1960;Cucerzan & Brill, 2004;Damerau & Mays, 1989;Davidson, 1962;Gadd, 1988;Kukich, 1992;Landau & Vishkin, 1986;Petersen, 1986;Pollock & Zamora, 1984;Rogers & Willett, 1991;Takahashi, Itoh, Amano, & Yamashita, 1990;Ullmann, 1977;Zobel & Dart, 1995). A number of studies have highlighted the context for correcting such errors and eliminating ambiguity (Damerau, 1964;Hull, 1992;Mays, Damerau, & Mercer, 1991;Riseman & Ehrich, 1971;Schulz & Mihov, 2002).…”

Section: Introductionmentioning

confidence: 99%

Approximate personal name‐matching through finite‐state graphs

Gálvez

Anegón

2007

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

This article shows how finite-state methods can be employed in a new and different task: the conflation of personal name variants in standard forms. In bibliographic databases and citation index systems, variant forms create problems of inaccuracy that affect information retrieval, the quality of information from databases, and the citation statistics used for the evaluation of scientists' work. A number of approximate string matching techniques have been developed to validate variant forms, based on similarity and equivalence relations. We classify the personal name variants as nonvalid and valid forms. In establishing an equivalence relation between valid variants and the standard form of its equivalence class, we defend the application of finite-state transducers. The process of variant identification requires the elaboration of: (a) binary matrices and (b) finite-state graphs. This procedure was tested on samples of author names from bibliographic records, selected from the Library and Information Science Abstracts and Science Citation Index Expanded databases. The evaluation involved calculating the measures of precision and recall, based on completeness and accuracy. The results demonstrate the usefulness of this approach, although it should be complemented with methods based on similarity relations for the recognition of spelling variants and misspellings.

show abstract

A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal errors in words

Cited by 106 publications

References 0 publications

Backwards Search in Context Bound Text Transformations

Backwards Search in Context Bound Text Transformations

Developing a Spell Checker as an Expert System

Approximate personal name‐matching through finite‐state graphs

Contact Info

Product

Resources

About