Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging

Abstract

We consider the construction of part-of-speech taggers for resource-poor languages. Recently, manually constructed tag dictionaries from Wiktionary and dictionaries projected via bitext have been used as type constraints to overcome the scarcity of annotated data in this setting. In this paper, we show that additional token constraints can be projected from a resource-rich source language to a resource-poor target language via word-aligned bitext. We present several models to this end; in particular a partially observed conditional random field model, where coupled token and type constraints provide a partial signal for training. Averaged across eight previously studied Indo-European languages, our model achieves a 25% relative error reduction over the prior state of the art. We further present successful results on seven additional languages from different families, empirically demonstrating the applicability of coupled token and type constraints across a diverse set of languages.

Neural Cross-Lingual Named Entity Recognition with Minimal Resources

“…Annotation projection methods create training data by using parallel corpora to project annotations from the source to the target language. Such approaches have been applied to many tasks under the cross-lingual setting, such as POS tagging (Yarowsky et al, 2001;Das and Petrov, 2011;Täckström et al, 2013;Fang and Cohn, 2016), mention detection (Zitouni and Florian, 2008) and parsing (Hwa et al, 2005;McDonald et al, 2011).…”

Choosing Transfer Languages for Cross-Lingual Learning

“…A common challenge in applying natural language processing (NLP) techniques to low-resource languages is the lack of training data in the languages in question. It has been demonstrated that through cross-lingual transfer, it is possible to leverage one or more similar high-resource languages to improve the performance on the low-resource languages in several NLP tasks, including machine score(L tf,1 , L tk ) score (L tf,2 , L tk translation (Zoph et al, 2016;Johnson et al, 2017;Nguyen and Chiang, 2017;Neubig and Hu, 2018), parsing (Täckström et al, 2012;Ammar et al, 2016;Ahmad et al, 2019;, partof-speech or morphological tagging (Täckström et al, 2013;Cotterell and Heigold, 2017;Malaviya et al, 2018;Plank and Agić, 2018), named entity recognition (Zhang et al, 2016;Mayhew et al, 2017;Xie et al, 2018), and entity linking (Tsai and Roth, 2016;Rijhwani et al, 2019). There are many methods for performing this transfer, including joint training (Ammar et al, 2016;Tsai and Roth, 2016;Cotterell and Heigold, 2017;Johnson et al, 2017;Malaviya et al, 2018), annotation projection (Täckström et al, 2012;Täckström et al, 2013;Zhang et al, 2016;Plank and Agić, 2018), fine-tuning (Zoph et al, 2016;Neubig and Hu, 2018), data augmentation (Mayhew et al, 2017), or zero-shot transfer (Ahmad et al, 2019;Xie et al, 2018;Neubig and Hu, 2018;Rijhwani et al, 2019).…”

Entity Projection via Machine Translation for Cross-Lingual NER

“…Cross-lingual approaches have been applied to many NLP tasks, including part-of-speech tagging (Yarowsky et al, 2001;Xi and Hwa, 2005;Das and Petrov, 2011;Täckström et al, 2013), parsing Zeman and Resnik, 2008;Smith and Eisner, 2009;Ganchev et al, 2009), and semantic role labeling (Tonelli and Pianta, 2008;Padó and Lapata, 2009;Titov, 2013, 2014). Prior cross-lingual NLP papers cleave roughly into two distinct approaches: direct model transfer and annotation projection.…”