Context Sensitive Lemmatization Using Two Successive Bidirectional Gated Recurrent Networks

Abstract

We introduce a composite deep neural network architecture for supervised and language independent context sensitive lemmatization. The proposed method considers the task as to identify the correct edit tree representing the transformation between a word-lemma pair. To find the lemma of a surface word, we exploit two successive bidirectional gated recurrent structures -the first one is used to extract the character level dependencies and the next one captures the contextual information of the given word. The key advantages of our model compared to the state-of-the-art lemmatizers such as Lemming and Morfette are -(i) it is independent of human decided features (ii) except the gold lemma, no other expensive morphological attribute is required for joint learning. We evaluate the lemmatizer on nine languages -Bengali, Catalan, Dutch, Hindi, Hungarian, Italian, Latin, Romanian and Spanish. It is found that except Bengali, the proposed method outperforms Lemming and Morfette on the other languages. To train the model on Bengali, we develop a gold lemma annotated dataset 1 (having 1, 702 sentences with a total of 20, 257 word tokens), which is an additional contribution of this work.

Improving Lemmatization of Non-Standard Languages with Joint Learning

“…More recently, Cotterell et al (2015) have used an extended set of features and a second-order CRF to jointly predict POS-tags and edit-trees with stateof-the-art performance. Finally, Chakrabarty et al (2017) employed a softmax classifier to predict edit-trees based on sentence-level features implicitly learned with a neural encoder over the input sentence.…”

Character-based recurrent neural networks for morphological relational reasoning

“…Lemmatization is the task of predicting the base form (lemma) of an inflected word. A lemmatizer may make use of the context to get (implicit) information about the source form of the word (Koskenniemi 1984; Kanis and Müller 2005;Chrupała et al 2008;Jongejan and Dalianis 2009;Chakrabarty et al 2017). In comparison, our task does not offer contextual information, but instead provides the (similarly implicit) cues for the forms from the demo relation.…”

Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection

“…The ISI system started with a heuristically induced candidate set, using the edit tree approach described by Chrupała et al (2008), and then chose the best edit tree. This approach is effectively a neuralized version of the lemmatizer proposed in Müller et al (2015) and, indeed, was originally intended for that task (Chakrabarty et al, 2017). The UA team, following their 2016 submission, proposed a linear reranking on top of the k-best output of their transduction system.…”

“…Our model is related to the encoder-decoder based approaches such as (Aharoni et al, 2016;Kann and Schütze, 2016a,b), but the main difference is that the proposed network is not designed to generate sequence of characters as output. Rather, we formulate the problem as to classify the transformation process required to convert a source form to its target form (Chakrabarty et al, 2017). Our goal is to model such a system which receives an input word and the morphological tags and returns the proper transformation that induces the target word.…”