An embedded segmental K-means model for unsupervised segmentation and clustering of speech

Abstract

Unsupervised segmentation and clustering of unlabelled speech are core problems in zero-resource speech processing. Most approaches lie at methodological extremes: some use probabilistic Bayesian models with convergence guarantees, while others opt for more efficient heuristic techniques. Despite competitive performance in previous work, the full Bayesian approach is difficult to scale to large speech corpora. We introduce an approximation to a recent Bayesian model that still has a clear objective function but improves efficiency by using hard clustering and segmentation rather than full Bayesian inference. Like its Bayesian counterpart, this embedded segmental K-means model (ES-KMeans) represents arbitrary-length word segments as fixed-dimensional acoustic word embeddings. We first compare ES-KMeans to previous approaches on common English and Xitsonga data sets (5 and 2.5 hours of speech): ES-KMeans outperforms a leading heuristic method in word segmentation, giving similar scores to the Bayesian model while being 5 times faster with fewer hyperparameters. However, its clusters are less pure than those of the other models. We then show that ES-KMeans scales to larger corpora by applying it to the 5 languages of the Zero Resource Speech Challenge 2017 (up to 45 hours), where it performs competitively compared to the challenge baseline. 1

Multilingual Acoustic Word Embedding Models for Processing Zero-resource Languages

“…Speech audio is parametrised as D = 13 dimensional static Mel-frequency cepstral coefficients (MFCCs). We use an embedding dimensionality of M = 130 throughout, since downstream systems such as the segmentation and clustering system of [8] are constrained to embedding sizes of this order. All encoder-decoder models have 3 encoder and 3 decoder unidirectional RNN layers, each with 400 units.…”

Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

“…For all models we use an embedding dimensionality of M = 130, to be directly comparable to the downsampling baseline. More importantly, although other studies consider higherdimensional settings, downstream systems such as [14] are constrained to embedding sizes of this order. Neural network architectures were optimised on the English validation data.…”

Measuring the perceptual availability of phonological features during language acquisition using unsupervised binary stochastic autoencoders

“…The present paper has a strong connection to recent work on unsupervised speech processing, especially the Zerospeech 2015 (Versteegh et al, 2015) and 2017 (Dunbar et al, 2017) shared tasks. Participating systems (Badino et al, 2015;Renshaw et al, 2015;Agenbag and Niesler, 2015;Baljekar et al, 2015;Räsänen et al, 2015;Lyzinski et al, 2015;Zeghidour et al, 2016;Heck et al, 2016;Srivastava and Shrivastava, 2016;Kamper et al, 2017b;Yuan et al, 2017;Heck et al, 2017;Shibata et al, 2017;Ansari et al, 2017a,b) perform unsupervised ABX discrimination and/or spoken term discovery on the basis of unlabeled speech alone. The design and evaluation of these and related systems (Kamper et al, , 2017aElsner and Shain, 2017;Räsänen et al, 2018) are oriented toward word-level modeling.…”