Tackling the Low-resource Challenge for Canonical Segmentation

Abstract

Canonical morphological segmentation consists of dividing words into their standardized morphemes. Here, we are interested in approaches for the task when training data is limited. We compare model performance in a simulated low-resource setting for the highresource languages German, English, and Indonesian to experiments on new datasets for the truly low-resource languages Popoluca and Tepehua. We explore two new models for the task, borrowing from the closely related area of morphological generation: an LSTM pointer-generator and a sequence-to-sequence model with hard monotonic attention trained with imitation learning. We find that, in the low-resource setting, the novel approaches outperform existing ones on all languages by up to 11.4% accuracy. However, while accuracy in emulated low-resource scenarios is over 50% for all languages, for the truly lowresource languages Popoluca and Tepehua, our best model only obtains 37.4% and 28.4% accuracy, respectively. Thus, we conclude that canonical segmentation is still a challenging task for low-resource languages.

An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers

“…Our study also relates to computational work on derivational morphology Cotterell and Schütze, 2018;Deutsch et al, 2018;Hofmann et al, 2020a,b,c) and word segmentation Kann et al, 2016;Ruzsics and Samardžić, 2017;Mager et al, 2019Mager et al, , 2020Seker and Tsarfaty, 2020;Amrhein and Sennrich, 2021). We are the first to systematically evaluate the segmentations of PLM tokenizers on human-annotated gold data.…”

SIGMORPHON 2022 Shared Task on Morpheme Segmentation Submission Description: Sequence Labelling for Word-Level Morpheme Segmentation

“…Morpheme segmentation is a well-established task in computational linguistics (cf. Mager et al (2020)). Recently, two definitions of morpheme segmentations have emerged: "Shallow segmentation" and "canonical segmentation" (Kann et al, 2016).…”

Quantifying Synthesis and Fusion and their Impact on Machine Translation

“…There are both supervised (e.g. pointer generator networks (Mager et al, 2020)) and unsupervised approaches (e.g. the Morfessor family of methods (Creutz and Lagus, 2002;Poon and Domingos, 2009) or Adaptor Grammars (Eskander et al, 2019)), where the former ones have outperformed the latter ones.…”

“…• Morfessor (Poon and Domingos, 2009). 4 We analysed several vocabulary sizes (4k, 8k, 16k, 32k, 64k) • Pointer Generator Network (PtrNet) from the implementation of Mager et al (2020).…”