Massively Multilingual Neural Grapheme-to-Phoneme Conversion

Peters, Ben; Dehdari, Jon; Genabith, Josef van

doi:10.18653/v1/w17-5403

Cited by 37 publications

(37 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Class We use grapheme-to-phoneme (G2P) as a proxy of a phonological task (Deri and Knight, 2016;Peters et al, 2017). The dataset contains over 650,000 such training instances, for a total of 311 languages (Deri and Knight, 2016).…”

Section: Datasetmentioning

confidence: 99%

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

Bjerva

Augenstein

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

A core part of linguistic typology is the classification of languages according to linguistic properties, such as those detailed in the World Atlas of Language Structure (WALS). Doing this manually is prohibitively time-consuming, which is in part evidenced by the fact that only 100 out of over 7,000 languages spoken in the world are fully covered in WALS.We learn distributed language representations, which can be used to predict typological properties on a massively multilingual scale. Additionally, quantitative and qualitative analyses of these language embeddings can tell us how language similarities are encoded in NLP models for tasks at different typological levels. The representations are learned in an unsupervised manner alongside tasks at three typological levels: phonology (grapheme-to-phoneme prediction, and phoneme reconstruction), morphology (morphological inflection), and syntax (part-of-speech tagging).We consider more than 800 languages and find significant differences in the language representations encoded, depending on the target task. For instance, although Norwegian Bokmål and Danish are typologically close to one another, they are phonologically distant, which is reflected in their language embeddings growing relatively distant in a phonological task. We are also able to predict typological features in WALS with high accuracies, even for unseen language families.

show abstract

Section: Datasetmentioning

confidence: 99%

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

Bjerva

Augenstein

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

show abstract

“…They also show evidence that the model can handle intrasentence code-switching. Peters et al (2017) train a multilingual sequence-to-sequence translation architecture on grapheme-to-phoneme conversion using more than 300 languages. They report better performance when adding multiple languages, even those which are not present in the test data.…”

Section: Related Workmentioning

confidence: 99%

Lessons Learned in Multilingual Grounded Language Learning

Kádár

Elliott²,

Côté³

et al. 2018

Proceedings of the 22nd Conference on Computational Natural Language Learning

View full text Add to dashboard Cite

Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions affect the performance of this type of grounded language learning model. We show that multilingual training improves over bilingual training, and that low-resource languages benefit from training with higher-resource languages. We demonstrate that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.

show abstract

“…For zero-shot learning (Sec. 3.5), we use an architecture very similar to Johnson et al (2016), also used for grapheme-tophoneme mapping in Peters et al (2017).…”

Section: Related Workmentioning

confidence: 99%

Few-Shot and Zero-Shot Learning for Historical Text Normalization

Bollmann

Korchagina

Søgaard

2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been no systematic study of different multitask learning architectures. This paper evaluates 63 multi-task learning configurations for sequence-to-sequence-based historical text normalization across ten datasets from eight languages, using autoencoding, grapheme-tophoneme mapping, and lemmatization as auxiliary tasks. We observe consistent, significant improvements across languages when training data for the target task is limited, but minimal or no improvements when training data is abundant. We also show that zero-shot learning outperforms the simple, but relatively strong, identity baseline.

show abstract

Massively Multilingual Neural Grapheme-to-Phoneme Conversion

Cited by 37 publications

References 17 publications

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

Lessons Learned in Multilingual Grounded Language Learning

Few-Shot and Zero-Shot Learning for Historical Text Normalization

Contact Info

Product

Resources

About