A Simple Joint Model for Improved Contextual Neural Lemmatization

Malaviya, Chaitanya; Wu, Shijie; Cotterell, Ryan

doi:10.18653/v1/n19-1155

Cited by 17 publications

(22 citation statements)

References 24 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A summary of the average results of each model configuration with a comparison to the baseline(Malaviya et al, 2019).…”

mentioning

confidence: 99%

Cross-Lingual Lemmatization and Morphology Tagging with Two-Stage Multilingual BERT Fine-Tuning

Kondratyuk¹

2019

Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology

View full text Add to dashboard Cite

We present our CHARLES-SAARLAND system for the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology, in task 2, Morphological Analysis and Lemmatization in Context. We leverage the multilingual BERT model and apply several fine-tuning strategies introduced by UDify demonstrating exceptional evaluation performance on morpho-syntactic tasks. Our results show that fine-tuning multilingual BERT on the concatenation of all available treebanks allows the model to learn cross-lingual information that is able to boost lemmatization and morphology tagging accuracy over fine-tuning it purely monolingually. Unlike UDify, however, we show that when paired with additional character-level and word-level LSTM layers, a second stage of fine-tuning on each treebank individually can improve evaluation even further. Out of all submissions for this shared task, our system achieves the highest average accuracy and f1 score in morphology tagging and places second in average lemmatization accuracy.

show abstract

“…A summary of the average results of each model configuration with a comparison to the baseline(Malaviya et al, 2019).…”

mentioning

confidence: 99%

Cross-Lingual Lemmatization and Morphology Tagging with Two-Stage Multilingual BERT Fine-Tuning

Kondratyuk¹

2019

Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology

View full text Add to dashboard Cite

show abstract

“…Neural (Malaviya et al, 2019): This is a stateof-the-art neural model that also performs joint morphological tagging and lemmatization, but also accounts for the exposure bias with the application of maximum likelihood (MLE). The model stitches the tagger and lemmatizer together with the use of jackknifing (Agić and Schluter, 2017) to expose the lemmatizer to the errors made by the tagger model during training.…”

Section: Task 2 Baselinesmentioning

confidence: 99%

The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

McCarthy¹,

Vylomova²,

Wu³

et al. 2019

Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology

Self Cite

View full text Add to dashboard Cite

The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as contextual lemmatization and morphosyntactic description in 66 languages. The first task evolves past years' inflection tasks by examining transfer of morphological inflection knowledge from a high-resource language to a low-resource language. This year also presents a new second challenge on lemmatization and morphological feature analysis in context. All submissions featured a neural component and built on either this year's strong baselines or highly ranked systems from previous years' shared tasks. Every participating team improved in accuracy over the baselines for the inflection task (though not Levenshtein distance), and every team in the contextual analysis task improved on both state-of-the-art neural and non-neural baselines. Data Data for Task 1Language pairs We presented data in 100 language pairs spanning 79 unique languages. Data for all but four languages (Basque, Kurmanji, Murrinhpatha, and Sorani) are extracted from English Wiktionary, a large multi-lingual crowdsourced dictionary with morphological paradigms

show abstract

“…We use the neural model from Malaviya et al (2019) for contextual lemmatization. This is a neural sequence-to-sequence model with hard attention, which takes both the inflected form and morphological tag set for a token as input and produces a lemma, both at the character level.…”

Section: Contextual Lemmatizationmentioning

confidence: 99%

“…The decoder uses the concatenation of the previous character and the tag set to produce the next character in the lemma. The lemmatization model is jointly trained with an LSTM-based tagger using jackknifing to reduce exposure bias in training: Malaviya et al (2019) report significantly lower lemmatization results training with gold tags and using predicted tags only at test time. We use their tagger for training and our contextual morphological analysis models' predicted tags at evaluation time.…”

Section: Contextual Lemmatizationmentioning

confidence: 99%

“…To address the issue of data scarcity, we present two multilingual transfer approaches where we train on a group of typologically related languages and find that language-groups with shallower time-depths (i.e., period of time during which languages diverged to become independent) tend to benefit the most from transfer. We focus on the task of contextual morphological analysis and use the provided baseline model for the task of lemmatization (Malaviya et al, 2019). This paper makes the following contributions: 1.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology

Chaudhary¹,

Salesky²,

Bhat³

et al. 2019

Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology

View full text Add to dashboard Cite

This paper presents the submission by the CMU-01 team to the SIGMORPHON 2019 task 2 of Morphological Analysis and Lemmatization in Context. This task requires us to produce the lemma and morpho-syntactic description of each token in a sequence, for 107 treebanks. We approach this task with a hierarchical neural conditional random field (CRF) model which predicts each coarse-grained feature (eg. POS, Case, etc.) independently. However, most treebanks are under-resourced, thus making it challenging to train deep neural models for them. Hence, we propose a multi-lingual transfer training regime where we transfer from multiple related languages that share similar typology. 1 Cohen. 2016.Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270.

show abstract

A Simple Joint Model for Improved Contextual Neural Lemmatization

Cited by 17 publications

References 24 publications

Cross-Lingual Lemmatization and Morphology Tagging with Two-Stage Multilingual BERT Fine-Tuning

Cross-Lingual Lemmatization and Morphology Tagging with Two-Stage Multilingual BERT Fine-Tuning

The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology

Contact Info

Product

Resources

About