Acoustic Segment Modeling with Spectral Clustering Methods

Abstract

This paper presents a study of spectral clustering-based approaches to acoustic segment modeling (ASM). ASM aims at finding the underlying phoneme-like speech units and building the corresponding acoustic models in the unsupervised setting, where no prior linguistic knowledge and manual transcriptions are available. A typical ASM process involves three stages, namely initial segmentation, segment labeling, and iterative modeling. This work focuses on the improvement of segment labeling. Specifically, we use posterior features as the segment representations, and apply spectral clustering algorithms on the posterior representations. We propose a Gaussian component clustering (GCC) approach and a segment clustering (SC) approach. GCC applies spectral clustering on a set of Gaussian components, and SC applies spectral clustering on a large number of speech segments. Moreover, to exploit the complementary information of different posterior representations, a multiview segment clustering (MSC) approach is proposed. MSC simultaneously utilizes multiple posterior representations to cluster speech segments. To address the computational problem of spectral clustering in dealing with large numbers of speech segments, we use inner product similarity graph and make reformulations to avoid the explicit computation of the affinity matrix and Laplacian matrix. We carried out two sets of experiments for evaluation. First, we evaluated the ASM accuracy on the OGI-MTS dataset, and it was shown that our approach could yield 18.7% relative purity improvement and 15.1% relative NMI improvement compared with the baseline approach. Second, we examined the performances of our approaches in the real application of zero-resource query-by-example spoken term detection on SWS2012 dataset, and it was shown that our approaches could provide consistent improvement on four different testing scenarios with three evaluation metrics.Index Terms-Acoustic segment modeling, multiview segment clustering, sub-word unit discovery, unsupervised training, zeroresource query-by-example spoken term detection.

Sparse Subspace Modeling for Query by Example Spoken Term Detection

“…To overcome these limitations, model-based approaches have been investigated [24], [33], [34]. These approaches primarily rely on acoustic units discovered in an unsupervised manner.…”

Improving Unsupervised Subword Modeling via Disentangled Speech Representation Learning and Transformation

“…Open-source tools [13] are used to train FHVAEs. 1 "speakers-R/-L" denotes speakers with rich/limited speech data. In our preliminary experiments, the ABX performance of z1 was found to be sensitive to the input segment length l. This could be explained as: a too large l would reduce the capability of z1 in modeling linguistic content at subword level; a too small l would restrict the FHVAE from capturing sufficient temporal dependencies which are essential in modeling speech.…”

“…UAM is a challenging problem with significant practical impact in speech as well as linguistics and cognitive science communities. It has been studied in applications such as ASR for low-resource languages [1], language identification [2] and query-by-example spoken term detection [3]. This problem is also relevant to endangered language protection [4] and understanding infants' language acquisition mechanism [5].…”

Comparison of ALBAYZIN query-by-example spoken term detection 2012 and 2014 evaluations

“…Unsupervised spoken term detection techniques, which aim at automatically discovering acoustic patterns (e.g., for training acoustic models) for languages for which manual transcriptions and linguistic knowledge are scarce, have been also investigated [34,35]. These techniques can also be employed for building language-independent QbE STD systems, since prior knowledge of the language is not necessary.…”