Neural morphosyntactic tagging for Rusyn

Abstract

The paper presents experiments on part-of-speech and full morphological tagging of the Slavic minority language Rusyn. The proposed approach relies on transfer learning and uses only annotated resources from related Slavic languages, namely Russian, Ukrainian, Slovak, Polish, and Czech. It does not require any annotated Rusyn training data, nor parallel data or bilingual dictionaries involving Rusyn. Compared to earlier work, we improve tagging performance by using a neural network tagger and larger training data from the neighboring Slavic languages. We experiment with various data preprocessing and sampling strategies and evaluate the impact of multitask learning strategies and of pretrained word embeddings. Overall, while genre discrepancies between training and test data have a negative impact, we improve full morphological tagging by 9% absolute micro-averaged F1 as compared to previous research.

Adaptation of Morphosyntactic Taggers

“…An ablation study furthermore showed that in agreement with linguistic knowledge, removing Ukrainian from the multi-source tagger hurts its performance most, whereas the contributions of Polish and Slovak are minor and the Russian corpora do not facilitate Rusyn tagging at all. Follow-up experiments with neural taggers on the same multi-source setup showed improvements of up to 5 percent absolute in terms of full morphological tagging accuracy (Scherrer and Rabus, 2019).…”

“…While most work is based on the first option, Müller et al (2013) and Pinter et al (2017) are notable representatives of the second option, the former relying on a CRF model, the latter on a recurrent neural network model. Our own recent experiments (Scherrer and Rabus, 2017;Scherrer et al, 2018;Scherrer and Rabus, 2019) use these two tagging architectures. A related approach has been proposed by Silfverberg and Drobac (2018), who view the tag string as a sequence that is generated one feature at a time.…”

“…This allows the tagger to take advantage of the increased training corpus size, but also to learn di↵erences between the varieties. Such an approach has been proposed by Cotterell and Heigold (2017) and has been applied successfully in Scherrer and Rabus (2019).…”

Natural language processing for similar languages, varieties, and dialects: A survey

“…In this case, information can be shared across similarly spelled words both within the same language variety (e.g., if there is no orthographic standard) as well as across language varieties (e.g, in a multilingual model). For example, Scherrer and Rabus (2019) represented the input words using a bidirectional character-level long short-term memory (LSTM) recurrent neural network and obtained up to 13% absolute boost in terms of F1-score compared to using atomic word-level representations. Zupan et al (2019) also stressed the importance of characterlevel input representations.…”

“…b The special issue received a record number of submissions for an NLE special issue and it was eventually split into two parts. The articles published in Part 1 (NLE 25:5) have demonstrated the vibrancy of research on the topic, covering a number of applications areas such as morphosyntactic tagging (Scherrer and Rabus 2019), text normalization (Martinc and Pollak 2019), and language identification (Jauhiainen, Lindén, and Jauhiainen 2019).…”