An effective algorithm for string correction using generalized edit distances—I. Description of the algorithm and its optimality

Kashyap, R.L.; Oommen, B. John

doi:10.1016/0020-0255(81)90052-9

Cited by 40 publications

(47 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first of these formally describes the module which accounts for the "dictionary" of ungarbled observations and the second describes the channel which syntactically corrupts error-free signals. This approach differs significantly from the original traditional way by which the problem was approached by the pioneer Fu 5 [3] and his co-authors. The latter represented the set of possible garbled occurrences of possible patterns using stochastic string grammars from the wellknown Chomsky hierarchy, or by web or tree grammars.…”

Section: Introductionmentioning

confidence: 97%

A formal theory for optimal and information theoretic syntactic pattern recognition

Oommen

Kashyap

1998

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

In this paper we present a foundational basis for optimal and information theoretic syntactic pattern recognition. We do this by developing a rigorous model, M * , for channels which permit arbitrarily distributed substitution, deletion and insertion syntactic errors. More explicitly, if A is any finite alphabet and A * the set of words over A, we specify a stochastically consistent scheme by which a string U ∈ A * can be transformed into any Y ∈ A * by means of arbitrarily distributed substitution, deletion and insertion operations. The scheme is shown to be Functionally Complete and stochastically consistent. Apart from the synthesis aspects, we also deal with the analysis of such a model and derive a technique by which Pr[Y|U], the probability of receiving Y given that U was transmitted, can be computed in cubic time using dynamic programming. One of the salient features of this scheme is that it demonstrates how dynamic programming can be applied to evaluate quantities involving complex combinatorial expressions and which also maintain rigid probability consistency constraints. Experimental results which involve dictionaries with strings of lengths between 7 and 14 with an overall average noise of 39.75 % demonstrate the superiority of our system over existing methods. Apart from its straightforward applications in string generation and recognition, we believe that the model also has extensive potential applications in speech, uni-dimensional signal processing, and computational molecular biology.

show abstract

Section: Introductionmentioning

confidence: 97%

A formal theory for optimal and information theoretic syntactic pattern recognition

Oommen

Kashyap

1998

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

show abstract

“…Also note that the same set of edit operations can be represented by multiple elements in Γ X,Y . This duplication serves as a powerful tool in the proofs of various analytic results [7,8,10,11,18].…”

Section: I1 Notationmentioning

confidence: 99%

“…Method : The dictionary consisted of 342 words obtained as a subset of the 1023 most common English words [7,9,12] augmented with words used in computer literature. The length of all the words in the dictionary was greater than or equal to 7 and the average length of a word was approximately 8.3 characters.…”

Section: Algorithm Produceeditoperations Inputmentioning

confidence: 99%

“…[1]. String correction using the GLD as a criterion has been done for strings [2,20,21], substrings [9], dictionaries treated as generalized tries [7] and for grammars [21]. Besides these deterministic techniques, various probabilistic methods have been studied in the literature [2,20,10].…”

Section: Introductionmentioning

confidence: 99%

“…Secondly, it permits the distances to be fairly arbitrary --they can be chosen to reflect the confusion matrix of the garbling mechanism --as is done in typical PR applications [7,17,21]. Last of all, (but if not the most important), is the relative simplicity of the present scheme --it is but a straightforward generalization of the Wagner-Fischer algorithm.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Pattern recognition of strings with substitutions, insertions, deletions and generalized transpositions

Oommen

Loke

1997

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

We study the problem of recognizing a string Y which is the noisy version of some unknown string X * chosen from a finite dictionary, H. The traditional case which has been extensively studied in the literature is the one in which Y contains substitution, insertion and deletion (SID) errors. Although some work has been done to extend the traditional set of edit operations to include the straightforward transposition of adjacent characters 2[14] the problem is unsolved when the transposed characters are themselves subsequently substituted, as is typical in cursive and typewritten script, in molecular biology and in noisy chain-coded boundaries. In this paper we present the first reported solution to the analytic problem of editing one string X to another, Y using these four edit operations. A scheme for obtaining the optimal edit operations has also been given. Both these solutions are optimal for the infinite alphabet case. Using these algorithms we present a syntactic pattern recognition scheme which corrects noisy text containing all these types of errors. The paper includes experimental results involving sub-dictionaries of the most common English words which demonstrate the superiority of our system over existing methods.1 Partially supported by the Natural Sciences and Engineering Research Council of Canada. A preliminary version of this paper can be found in the Proceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics, Vancouver, October 1995, pp. 1154-1159 There has been some recent work done [Oo93] to consider the squashing and expansion operations too, where in the squashing operation two (or more) contiguous characters of X can be transformed into a single character of Y, and in the expansion operation a single character in X may be expanded into two or more contiguous characters of Y.Pat. Recog. with Subst., Insert., Delet.and Gen. Transpos. Errors. Page 2

show abstract

Collection and characterization of spelling errors in scientific and scholarly text

Pollock¹,

Zamora²

1983

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

The SPEEDCOP (SPElling Error Detection Correction Project) project recently completed at Chemical Abstracts Service (CAS) extracted over 50,000 misspellings from approximately 25,000,000 words of text from seven scientific and scholarly databases. The misspellings were automatically classified and the error types analyzed. The results, which were consistent over the different databases, showed that the expected incidence of misspelling is 0.2%, that 90-95% of spelling errors have only a single mistake, that substitution is homogeneous while transposition is heterogeneous, that omission is the commonest type of misspelling, and that inadvertent doubling of a letter is the most important cause of insertion errors. The more frequently a letter occurs in the text, the more likely it is to be involved in a spelling error. Most misspellings collected by SPEEDCOP are of the type colloquially referred to as "typos" and approximately 90% are unlikely to be repeated in normal spans of text.

show abstract

An effective algorithm for string correction using generalized edit distances—I. Description of the algorithm and its optimality

Cited by 40 publications

References 18 publications

A formal theory for optimal and information theoretic syntactic pattern recognition

A formal theory for optimal and information theoretic syntactic pattern recognition

Pattern recognition of strings with substitutions, insertions, deletions and generalized transpositions

Collection and characterization of spelling errors in scientific and scholarly text

Contact Info

Product

Resources

About