Morphological Disambiguation of Turkish Text with Perceptron Algorithm

Abstract

Abstract. This paper describes the application of the perceptron algorithm to the morphological disambiguation of Turkish text. Turkish has a productive derivational morphology. Due to the ambiguity caused by complex morphology, a word may have multiple morphological parses, each with a different stem or sequence of morphemes. The methodology employed is based on ranking with perceptron algorithm which has been successful in some NLP tasks in English. We use a baseline statistical trigram-based model of a previous work to enumerate an n-best list of candidate morphological parse sequences for each sentence. We then apply the perceptron algorithm to rerank the n-best list using a set of 23 features. The perceptron trained to do morphological disambiguation improves the accuracy of the baseline model from 93.61% to 96.80%. When we train the perceptron as a POS tagger, the accuracy is 98.27%. Turkish morphological disambiguation and POS tagging results that we obtained is the best reported so far.

A Novel Approach to Morphological Disambiguation for Turkish

“…3 We used 10 different classification algorithms in the WEKA toolkit. The results of the classification algorithms and the previous approaches are given in Table 3.…”

“…The Perceptron Algorithm [3] is a combination of statistical and machine learning approaches. They use the Baseline Trigram-Based Model to generate n-best parses for each sentence.…”

“…To cope with the data sparseness problem, morphological parses are divided into smaller parts called inflectional groups [2]. The most recent approach to the morphological disambiguation problem is presented in [3]. The methodology employed is based on ranking of the most possible parse sequences (determined by the baseline statistical model represented in [1]) with Perceptron algorithm.…”

Morphological Disambiguation and Text Normalization for Southern Quechua Varieties

“…Finally, we generate the normalized word forms from the now disambiguated sequence of morphemes. Our initial results are comparable to morphological disambiguation on Turkish texts, despite the fact that we have a much smaller training corpus (∼ 2800 sentences, compared to over 50,000 (Görgün and Yildiz, 2011) and 45,000 sentences (Sak et al, 2007)). A possible explanation is that Turkish morphology is more complex: Turkish has more productive suffixes than Quechua, and there are relatively complex morpho-phonological rules that determine word formation, such as two dimensional vowel harmony and context-sensitive realizations of consonants (Oflazer, 1994).…”

Morphological Annotation of a Corpus with a Collaborative Multiplayer Game

“…There are also several constraint-based methods for disambiguation [18,19]. Another method employs a perceptron algorithm for morphological disambiguation [20]. We use the tool produced by this study as a morphological parser ranging from preparing the corpus to the online question generation.…”