Probabilistic Typology: Deep Generative Models of Vowel Inventories

Abstract

Linguistic typology studies the range of structures present in human language. The main goal of the field is to discover which sets of possible phenomena are universal, and which are merely frequent. For example, all languages have vowels, while most-but not all-languages have an [u] sound. In this paper we present the first probabilistic treatment of a basic question in phonological typology: What makes a natural vowel inventory? We introduce a series of deep stochastic point processes, and contrast them with previous computational, simulation-based approaches. We provide a comprehensive suite of experiments on over 200 distinct languages.

A Probabilistic Generative Model of Linguistic Typology

“…However, our work differs on several important counts, as we (i) include language information obtained through unsupervised learning, which allows us to take advantage of raw data and predict features for completely unannotated languages, (ii) analyse the effects of varying amounts of known features, especially in situations with and without in-branch training data, and (iii) view the problem of typological features through the lens of parameters from principles and parameters (Chomsky, 2000). Deep generative models have also been explored previously for modelling phonology (Cotterell and Eisner, 2017). Our work builds on these research directions, by (i) developing a deep generative model which (ii) takes advantage of correlations, rather than predicting features individually, and (iii) exploits unlabelled data.…”

DeepMove: Learning Place Representations through Large Scale Movement Data

“…ese representations improve the performance of machine translation system, and achieve positive results on some text classi cation and sentiment analysis tasks. ere are more and more applications of word embeddings in not only natural language tasks [8,16], but also in computer vision, e.g., Karpathy and Fei-Fei [15] introduce a model that aligns parts of visual and language modalities to generate natural language descriptions of image regions based on weak labels in form of a dataset of images and sentences.…”

A Deep Generative Model of Vowel Formant Typology

“…Before delving into our generative model, we briefly review technical background used by Cotterell and Eisner (2017). A DPP is a probability distribution over the subsets of a fixed ground set of size N -in our case, the set of phonesV.…”

“…We relate this latent space to measurable acoustic space by a learned diffeomorphism ν θ (Cotterell and Eisner, 2017). ν −1 θ can be regarded as warping the acoustic distances into perceptual/articulatory distances.…”

“…In this work, we offer a more formal treatment of the subject, deriving a generative probability model of vowel inventory typology. Our work builds on (Cotterell and Eisner, 2017) by investigating not just discrete IPA inventories but the cross-linguistic variation in acoustic formants.…”