A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages

Abstract

Unsupervised part of speech (POS) tagging is often framed as a clustering problem, but practical taggers need to ground their clusters as well. Grounding generally requires reference labeled data, a luxury a low-resource language might not have. In this work, we describe an approach for low-resource unsupervised POS tagging that yields fully grounded output and requires no labeled training data. We find the classic method of Brown et al. (1992) clusters well in our use case and employ a decipherment-based approach to grounding. This approach presumes a sequence of cluster IDs is a 'ciphertext' and seeks a POS tag-tocluster ID mapping that will reveal the POS sequence. We show intrinsically that, despite the difficulty of the task, we obtain reasonable performance across a variety of languages. We also show extrinsically that incorporating our POS tagger into a name tagger leads to stateof-the-art tagging performance in Sinhalese and Kinyarwanda, two languages with nearly no labeled POS data available. We further demonstrate our tagger's utility by incorporating it into a true 'zero-resource' variant of the MALOPA (Ammar et al., 2016) dependency parser model that removes the current reliance on multilingual resources and gold POS tags for new languages. Experiments show that including our tagger makes up much of the accuracy lost when gold POS tags are unavailable.

Grounding ‘Grounding’ in NLP

“…• Textual Modality: Within text, there are several datasets for tasks like content transfer (Prabhumoye et al, 2019), commonsense inference (Zellers et al, 2018), reference resolution (Kennington and Schlangen, 2015), symbol grounding (Kameko et al, 2015), studying linguistic and non-linguistic contexts in microblogs (Doyle and Frank, 2015), bilingual lexicon extraction (Laws et al, 2010), universal part-of-speech tagging for low resource languages (Cardenas et al, 2019), entity linking and reference (Nothman et al, 2012) etc.,…”

“…This is not an exhaustive study of all the techniques that present grounding, but are some of the representative categories. Here are more studies that perform grounding with various techniques such as clustering (Shutova et al, 2015;Cardenas et al, 2019) regularization (Shrestha et al, 2020), CRFs , classification (Pangburn et al, 2003;Monroe et al, 2017), linguistic theories (Strube and Hahn, 1999), iterative refinement (Li et al, 2019;, language modeling (Spithourakis et al, 2016;Cho and May, 2020), nearest neighbors , contextual fusion (Chandu et al, 2019a), mutual information (Oates, 2003), cycle consistency (Zhong et al, 2020) etc.,…”

Finite-state script normalization and processing utilities: The Nisaba Brahmic library

“…The main fo cus of these libraries is script conversion and ro manization. In this capacity they were success fully employed in diverse downstream multilin gual NLP tasks such as neural machine transla tion (Zhang et al, 2020; Amrhein andSennrich, 2020), morphological analysis (Hauer et al, 2019; Murikinati et al, 2020, named entity recogni tion (Huang et al, 2019) and partofspeech tag ging (Cardenas et al, 2019).…”

Optimal Size-Performance Tradeoffs: Weighing PoS Tagger Models

“…Another form of resource-constrained NLP is with low-resource languages or datasets. Research into unsupervised PoS tagging on low resource languages has been done by Buys and Botha (2016); Cardenas et al (2019). Ezen-Can (2020) evaluated the performance of BERT on a small dataset.…”