Automatic interlinear glossing as two-level sequence classification

Abstract

Interlinear glossing is a type of annotation of morphosyntactic categories and crosslinguistic lexical correspondences that allows linguists to analyse sentences in languages that they do not necessarily speak. Automatising this annotation is necessary in order to provide glossed corpora big enough to be used for quantitative studies. In this paper, we present experiments on the automatic glossing of Chintang. We decompose the task of glossing into steps suitable for statistical processing. We first perform grammatical glossing as standard supervised part-of-speech tagging. We then add lexical glosses from a stand-off dictionary applying context disambiguation in a similar way to word lemmatisation. We obtain the highest accuracy score of 96% for grammatical and 94% for lexical glossing.

Does morphological complexity affect word segmentation? Evidence from computational modeling

“…As for the morphological annotation, for Chintang, most of the corpus was hand-segmented and manually annotated for morphology and parts-of-speech by trained linguistic students under supervision by an expert in the language. A small part of the morphological annotation was generated automatically based on a morphological tagger (Ruzsics & Samardzic, 2017;Samardzic, Schikowski & Stoll, 2015). Japanese morphological tagging was done with the morphological tagger in CHILDES (JMOR, Miyata and Naka, 2014).…”

To POS Tag or Not to POS Tag: The Impact of POS Tags on Morphological Learning in Low-Resource Settings

“…The low-resource language data came from interlinearized data that was polished for publication. McMillan-Major (2020) and some other experiments such as Samardzic et al (2015) use information from lines of interlinearized texts such as translation and POS tags.…”

Automatic Interlinear Glossing for Under-Resourced Languages Leveraging Translations

“…• Accuracy: percentage of correct (full) analyses for each token. It is the main metric used in previous work (Samardžić et al, 2015;McMillan-Major, 2020).…”

“…Several works have studied the automated IGT generation task Samardžić et al, 2015;Moeller and Hulden, 2018;McMillan-Major, 2020). They mainly used machine learning methods such as CRF and SVM to generate gloss and proposed a series of heuristic post-editing algorithms to improve the performance.…”

“…stems (Samardžić et al, 2015), and using models based on Conditional Random Fields (CRF) integrated with translation and POS-tagging information (McMillan-Major, 2020). In contrast, our approach is, to our knowledge, the first one to show that modern neural systems are a viable solution for the automatic glossing task, without requiring any additional components or making unrealistic assumptions regarding data or NLP tool availability for low-resource languages.…”