Morphological Analysis Using a Sequence Decoder

Abstract

We introduce Morse, a recurrent encoder-decoder model that produces morphological analyses of each word in a sentence. The encoder turns the relevant information about the word and its context into a fixed size vector representation and the decoder generates the sequence of characters for the lemma followed by a sequence of individual morphological features. We show that generating morphological features individually rather than as a combined tag allows the model to handle rare or unseen tags and to outperform whole-tag models. In addition, generating morphological features as a sequence rather than, for example, an unordered set allows our model to produce an arbitrary number of features that represent multiple inflectional groups in morphologically complex languages. We obtain state-of-the-art results in nine languages of different morphological complexity under low-resource, high-resource, and transfer learning settings. We also introduce TrMor2018, a new high-accuracy Turkish morphology data set. Our Morse implementation and the TrMor2018 data set are available online to support future research. 1 See https://github.com/ai-ku/Morse.jl for a Morse implementation in Julia/Knet (Yuret, 2016 ) and https://github.com/ai-ku/TrMor2018 for the new Turkish data set.

Computational Morphology with Neural Network Approaches

“…Another work to jointly learn lemmatization and morphological tagging is Akyürek et al (2019). This system focuses on morphological tagging and the tagging results outperform Cotterell and Heigold (2017) and Malaviya et al (2018).…”

“…Neural models have been very successful in this aspect (Wolf-Sonkin et al, 2018). In addition, neural models have been more flexible and creative, and thus can deal with unseen words or MSD tags as well as ambiguous words much more effectively (Bergmanis and Goldwater, 2018;Akyürek et al, 2019).…”

“…Tagging tasks have seen improvements in models for low-resource languages when models are trained with related high-resource languages (e.g. Buys and Botha (2016), Cotterell and Heigold (2017), Akyürek et al (2019), and Kondratyuk ( 2019)), while high-resource languages are found to contribute to the (re)inflection models for low-resource languages in some work (e.g. , Jin and Kann (2017), Anastasopoulos and Neubig (2019)), and don't in some others (e.g.…”

Learning to Recombine and Resample Data for Compositional Generalization

“…1). We use the data to construct a morphological analysis task (Akyürek et al, 2019) finer-grained analysis of novel word forms: words in the evaluation set whose exact morphological analysis never appeared in the training set. Resampling and learned recombination again significantly outperform both the baseline and GECA-based data augmentation in the few-shot FUT+PAST condition and the ordinary OTHER condition, underscoring the effectiveness of this approach for "in-distribution" compositional generalization.…”

Joint learning of morphology and syntax with cross-level contextual information flow

“…As we shall see later, in Figures 9(c) and 10, in many cases looking at the previous word is not enough to disambiguate the possessor even in examples where we can determine that the shape has possession semantics; therefore, decoding on this basis alone would not be sufficient (cf. Akyürek, Dayanık, and Yuret 2019). It needs an attention mechanism or its functional equivalent.…”