Unsupervised discovery of linguistic structure including two-level acoustic patterns using three cascaded stages of iterative optimization

Abstract

Techniques for unsupervised discovery of acoustic patterns are getting increasingly attractive, because huge quantities of speech data are becoming available but manual annotations remain hard to acquire. In this paper, we propose an approach for unsupervised discovery of linguistic structure for the target spoken language given raw speech data. This linguistic structure includes two-level (subwordlike and word-like) acoustic patterns, the lexicon of word-like patterns in terms of subword-like patterns and the N-gram language model based on word-like patterns. All patterns, models, and parameters can be automatically learned from the unlabelled speech corpus. This is achieved by an initialization step followed by three cascaded stages for acoustic, linguistic, and lexical iterative optimization. The lexicon of word-like patterns defines allowed consecutive sequence of HMMs for subword-like patterns. In each iteration, model training and decoding produces updated labels from which the lexicon and HMMs can be further updated. In this way, model parameters and decoded labels are respectively optimized in each iteration, and the knowledge about the linguistic structure is learned gradually layer after layer. The proposed approach was tested in preliminary experiments on a corpus of Mandarin broadcast news, including a task of spoken term detection with performance compared to a parallel test using models trained in a supervised way. Results show that the proposed system not only yields reasonable performance on its own, but is also complimentary to existing large vocabulary ASR systems.Index Termsunsupervised learning, hidden Markov models, spoken term detection, zero resource speech recognition, iterative optimization i arXiv:1509.02208v1 [cs.CL] 7 Sep 2015

Acoustic Segment Modeling with Spectral Clustering Methods

“…Take inner product as the similarity measure, (24) Similar to (15), the computation of Laplacian matrices in (18) becomes, (25) where, (26) with . Then problem (20) is rewritten as,…”

“…With this approach, the number of speech units can be estimated automatically. In [26], a three-stage approach involving word-level pattern construction and word-level decoding was proposed. In [6] [27], the problem of unsupervised acoustic modeling was tackled by first discovering large-size units (e.g., words), and performing Gaussian component clustering with top-down constraints.…”

“…In [9]- [26], segment labeling is performed by vector quantization (VQ). Each segment is represented by the mean of frame-level feature vectors, and the -mean algorithm is used for clustering of the mean feature vectors.…”

Towards weakly supervised acoustic subword unit discovery and lexicon development using hidden Markov models

“…Alternately, the lexicon development process is weakly-supervised similar to acoustic model development in an ASR system. More recently, in the context of "zero-resourced" ASR system development, there are efforts towards developing methods that are fully unsupervised (Chung et al, 2013;Lee et al, 2015). Such methods are at very early stages and are out of the scope of this paper.…”

Gate Activation Signal Analysis for Gated Recurrent Neural Networks and its Correlation with Phoneme Boundaries

“…which is actually parallel to (11). The superscript n indicates the n-th training utterance and d indicates the number of dimensions of x n t .…”