Finite-State Technology as a Programming Environment

Abstract

Abstract. Finite-state technology is considered the preferred model for representing the phonology and morphology of natural languages. The attractiveness of this technology for natural language processing stems from four sources: modularity of the design, due to the closure properties of regular languages and relations; the compact representation that is achieved through minimization; efficiency, which is a result of linear recognition time with finite-state devices; and reversibility, resulting from the declarative nature of such devices.However, when wide-coverage grammars are considered, finite-state technology does not scale up well, and the benefits of this technology can be overshadowed by the limitations it imposes as a programming environment for language processing. This paper focuses on several aspects of large-scale grammar development. Using a real-world benchmark, we compare a finite-state implementation with an equivalent Java program with respect to ease of development, modularity, maintainability of the code and space and time efficiency. We identify two main problems, abstraction and incremental development, which are currently not addressed sufficiently well by finite-state technology, and which we believe should be the focus of future research and development.

Strengths and weaknesses of finite-state technology: a case study in morphological grammar development

“…This paper extends and revises an earlier version (Wintner 2007). This research was supported by The Israel Science Foundation (grant No.…”

Language resources for Hebrew

“…Several problems were encountered during the development and maintenance of the grammar, including poor compile-time performance, unreasonable memory requirements and lack of abstraction which resulted in maintenance difficulties (Wintner, 2007). Consequently, we reimplemented the analyzer in Java.…”

“…They have been extensively used for both research and commercial applications in the past few years, and are regularly maintained and supported. While parts of this project have been presented elsewhere (Wintner and Yona, 2003;Yona and Wintner, 2005;Bar-Haim et al, 2005;Itai, 2006;Adler and Elhadad, 2006;Itai et al, 2006;Wintner, 2007;Yona and Wintner, 2007;Ordan and Wintner, 2007), this paper provides a general framework for the full-scale project, extending, updating and elaborating on previous discussions.…”

Knowledge representation of grammatical constructs of Sanskrit Language and modular architecture of ParGram

“…II LITRETURE REVIEW Various studies have been done for FST ,Shuly Wintner focuses on several aspects of large-scale grammar development, and compare a finite-state implementation with an equivalent Java program with respect to ease of development, modularity, maintainability of thecode and space and time efficiency [1]. Austin, Peter K. introduced how to set C-structure and f-structure in the English language [2].…”

“…The first step is split the sentence in the tokens i.e., tokenization and morphological analysis, It is possible for using state-of-the-art finite state technology (FST) [1]. The output of the morphological analyzer feed as a input into the syntactic component, where the analyses are informed by the theoretical linguistic work.…”