A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages

Vania, Clara; Kementchedjhieva, Yova; Søgaard, Anders; Lopez, Adam

doi:10.18653/v1/d19-1102

Cited by 44 publications

(37 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As expected these scores are considerably lower than when using in-language OOD data, being so poor that these parsers are hardly useful, confirming previous research, e.g. Meechan-Maddon and Nivre (2019) and Vania et al (2019). In this case there is no clear difference between IND and OOD data.…”

Section: Resultssupporting

confidence: 86%

Cross-Lingual Domain Adaptation for Dependency Parsing

Stymne¹

2020

Proceedings of the 19th International Workshop on Treebanks and Linguistic Theories

View full text Add to dashboard Cite

We show how we can adapt parsing to low-resource domains by combining treebanks across languages for a parser model with treebank embeddings. We demonstrate how we can take advantage of in-domain treebanks from other languages, and show that this is especially useful when only out-of-domain treebanks are available for the target language. The method is also extended to low-resource languages by using out-of-domain treebanks from related languages. Two parameter-free methods for applying treebank embeddings at test time are proposed, which give competitive results to tuned methods when applied to Twitter data and transcribed speech. This gives us a method for selecting treebanks and training a parser targeted at any combination of domain and language.

show abstract

Section: Resultssupporting

confidence: 86%

Cross-Lingual Domain Adaptation for Dependency Parsing

Stymne¹

2020

Proceedings of the 19th International Workshop on Treebanks and Linguistic Theories

View full text Add to dashboard Cite

show abstract

“…In the computer vision community, this is a popular approach where, e.g., rotating an image is invariant to the classification of an image's content. For text, on the token level, this can be done by replacing words with equivalents, such as synonyms (Wei and Zou, 2019), entities of the same type (Raiman and Miller, 2017;Dai and Adel, 2020) or words that share the same morphology (Gulordava et al, 2018;Vania et al, 2019). Such replacements can also be guided by a language model that takes context into consideration (Fadaee et al, 2017;Kobayashi, 2018).…”

Section: Data Augmentationmentioning

confidence: 99%

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios

Hedderich¹,

Lange²,

Adel³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

136

View full text Add to dashboard Cite

Deep neural networks and huge language models are becoming omnipresent in natural language applications. As they are known for requiring large amounts of training data, there is a growing body of work to improve the performance in low-resource settings. Motivated by the recent fundamental changes towards neural models and the popular pre-train and fine-tune paradigm, we survey promising approaches for low-resource natural language processing. After a discussion about the different dimensions of data availability, we give a structured overview of methods that enable learning when training data is sparse. This includes mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific low-resource setting. Further key aspects of this work are to highlight open issues and to outline promising directions for future research.

show abstract

“…Toxic language classification has been conducted in a number of studies (Schmidt and Wiegand, 2017;Davidson et al, 2017;Wulczyn et al, 2017;Gröndahl et al, 2018;Qian et al, 2019;Breitfeller et al, 2019). NLP applications of data augmentation include text classification (Ratner et al, 2017;Wei and Zou, 2019;Mesbah et al, 2019), user behavior categorization (Wang and Yang, 2015), dependency parsing (Vania et al, 2019), and machine translation (Fadaee et al, 2019;Xia et al, 2019). Related techniques are also used in automatic paraphrasing (Madnani and Dorr, 2010;Li et al, 2018) and writing style transfer (Shen et al, 2017;Shetty et al, 2018;Mahmood et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

A little goes a long way: Improving toxic language classification despite data scarcity

Juuti¹,

Gröndahl²,

Flanagan

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Detection of some types of toxic language is hampered by extreme scarcity of labeled training data. Data augmentation -generating new synthetic data from a labeled seed datasetcan help. The efficacy of data augmentation on toxic language classification has not been fully explored. We present the first systematic study on how data augmentation techniques impact performance across toxic language classifiers, ranging from shallow logistic regression architectures to BERT -a state-of-the-art pretrained Transformer network. We compare the performance of eight techniques on very scarce seed datasets. We show that while BERT performed the best, shallow classifiers performed comparably when trained on data augmented with a combination of three techniques, including GPT-2-generated sentences. We discuss the interplay of performance and computational overhead, which can inform the choice of techniques under different constraints.

show abstract

A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages

Cited by 44 publications

References 27 publications

Cross-Lingual Domain Adaptation for Dependency Parsing

Cross-Lingual Domain Adaptation for Dependency Parsing

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios

A little goes a long way: Improving toxic language classification despite data scarcity

Contact Info

Product

Resources

About