Transducers from rewrite rules with backreferences

Gerdemann, Dale; Noord, Gertjan van

doi:10.3115/977035.977053

Cited by 18 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 Several authors have given recipes for finite-state transducers that perform a single contextual edit operation (Kaplan and Kay, 1994;Mohri and Sproat, 1996;Gerdemann and van Noord, 1999). Such "rewrite rules" can be individually more expressive than our simple edit operations of section 2; but it is unclear how to train a cascade of them to model p(y | x).…”

Section: Probabilistic Fstsmentioning

confidence: 99%

Stochastic Contextual Edit Distance and Probabilistic FSTs

Cotterell

Peng

Eisner

2014

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

String similarity is most often measured by weighted or unweighted edit distance d(x, y). Ristad and Yianilos (1998) defined stochastic edit distance-a probability distribution p(y | x) whose parameters can be trained from data. We generalize this so that the probability of choosing each edit operation can depend on contextual features. We show how to construct and train a probabilistic finite-state transducer that computes our stochastic contextual edit distance. To illustrate the improvement from conditioning on context, we model typos found in social media text.

show abstract

Section: Probabilistic Fstsmentioning

confidence: 99%

Stochastic Contextual Edit Distance and Probabilistic FSTs

Cotterell

Peng

Eisner

2014

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

show abstract

“…Here α, β, λ and ρ are regular expressions, which implies that in general we have a proper rewriting relation (as opposed to a rewriting function). For the leftmost-longest match strategy, the construction described in Kaplan and Kay (1994) has been improved by Gerdemann and van Noord (1999). Kaplan and Kay (1994) proved that all these rewriting relations are rational, i.e., they can be encoded by a finite state transducer.…”

Section: Related Workmentioning

confidence: 99%

Efficient dictionary-based text rewriting using subsequential transducers

Mihov

Schulz²

2007

Nat. Lang. Eng.

View full text Add to dashboard Cite

Problems in the area of text and document processing can often be described as text rewriting tasks: given an input text, produce a new text by applying some fixed set of rewriting rules. In its simplest form, a rewriting rule is given by a pair of strings, representing a source string (the "original") and its substitute. By a rewriting dictionary, we mean a finite list of such pairs; dictionary-based text rewriting means to replace in an input text occurrences of originals by their substitutes. We present an efficient method for constructing, given a rewriting dictionary D, a subsequential transducer T that accepts any text t as input and outputs the intended rewriting result under the so-called "leftmost-longest match" replacement with skips, t . The time needed to compute the transducer is linear in the size of the input dictionary. Given the transducer, any text t of length |t| is rewritten in a deterministic manner in time O(|t| + |t |), where t denotes the resulting output text. Hence the resulting rewriting mechanism is very efficient. As a second advantage, using standard tools, the transducer can be directly composed with other transducers to efficiently solve more complex rewriting tasks in a single processing step.

show abstract

“…These toolboxes also include efficient implementations of several standard algorithms on finite state machines, such as union, intersection, minimization, determinization etc. More importantly, they also implement special operators that are useful for linguistic description, such as replacement (Kaplan and Kay, 1994;Mohri and Sproat, 1996;Karttunen, 1997;Gerdemann and van Noord, 1999) or predicates over alphabet symbols (van Noord and Gerdemann, 2001a;van Noord and Gerdemann, 2001b), and even operators for particular linguistic theories such as Optimality Theory (Karttunen, 1998;Gerdemann and van Noord, 2000). Unfortunately, there are no standards for the syntax of extended regular expression languages and switching from one tool-box to another is a non-trivial task.…”

Section: Introductionmentioning

confidence: 99%

xfst2fsa

Cohen-Sygal

Wintner

2005

Proceedings of the Workshop on Software - Software '05

View full text Add to dashboard Cite

This paper introduces xfst2fsa, a compiler which translates grammars expressed in the syntax of the XFST finite-state toolbox to grammars in the language of the FSA Utilities package. Compilation to FSA facilitates the use of grammars developed with the proprietary XFST toolbox on a publicly available platform. The paper describes the non-trivial issues of the compilation process, highlighting several shortcomings of some published algorithms, especially where replace rules are concerned. The compiler augments FSA with most of the operators supported by XFST. Furthermore, it provides a means for comparing the two systems on comparable grammars. The paper presents the results of such a comparison.

show abstract

Transducers from rewrite rules with backreferences

Cited by 18 publications

References 12 publications

Stochastic Contextual Edit Distance and Probabilistic FSTs

Stochastic Contextual Edit Distance and Probabilistic FSTs

Efficient dictionary-based text rewriting using subsequential transducers

xfst2fsa

Contact Info

Product

Resources

About