Investigations on exemplar-based features for speech recognition towards thousands of hours of unsupervised, noisy data

Abstract

The acoustic models in state-of-the-art speech recognition systems are based on phones in context that are represented by hidden Markov models. This modeling approach may be limited in that it is hard to incorporate long-span acoustic context. Exemplar-based approaches are an attractive alternative, in particular if massive data and computational power are available. Yet, most of the data at Google are unsupervised and noisy. This paper investigates an exemplar-based approach under this yet not well understood data regime. A log-linear rescoring framework is used to combine the exemplar-based features on the word level with the first-pass model. This approach guarantees at least baseline performance and focuses on the refined modeling of words with sufficient data. Experimental results for the Voice Search and the YouTube tasks are presented.Index Terms-Exemplar-based speech recognition, conditional random fields, speech recognition INTRODUCTIONState-of-the-art speech recognition systems are based on hidden Markov models (HMMs) to represent phones in context. These models are convenient due to their simplicity and compactness. However, it is hard to incorporate long-span acoustic context into this type of models, without pooling observations from different examples on the frame level.Non-parametric, exemplar-based approaches such as knearest neighbors (kNN) appear to be an attractive alternative to overcome this limitation of conventional HMMs and may be more effective at capturing the large variability of speech. In this paper, we investigate an exemplar-based (also known as template-based) rescoring approach to speech recognition, which can be considered a variant of kNN on (pre-)segmented acoustic units such as words.Like for most non-parametric approaches, the main concerns about exemplar-based speech recognition are that it requires large amounts of data and thus, massive computational power. The origin of the complexity is twofold. First, there is no compact representation as in case of conventional HMMs and all data need to be memorized and processed. Second, the Dynamic Time Warping (DTW) distance [1, 2] is used to measure the similarity between two templates. Using

Discriminative acoustic word embeddings: Tecurrent neural network-based approaches

“…Whole-word approaches typically involve, at some level, template matching. For example, in template-based speech recognition [4,5], word scores are computed from dynamic time warping (DTW) distances between an observed segment and training segments of the hypothesized word. In queryby-example search, putative matches are typically found by measuring the DTW distance between the query and segments of the search database [6,7,8,9].…”

Integrated exemplar-based template matching and statistical modeling for continuous speech recognition

“…This data pruning reduced the database size or the model size by about 30%, and consequently saved the computation time and memory usage in speech recognition. In [21], exemplarbased word-level features were investigated for largescale speech recognition. These features were combined with the acoustic and language scores of the first-pass model through a segmental conditional random field to rescore word lattices.…”

“…However, they are still difficult to use in large vocabulary continuous speech recognition (LVCSR) due to their needs for intensive computing time and storage space. The newly proposed methods, such as template pruning and filtering [19], template-like dimension reduction of speech observations [20], and template matching in the second-pass decoding search [21], are beginning to address this problem. In general, there is a tradeoff between the costs in computation and space and the accuracy in recognition.…”

Noise robust exemplar matching with alpha–beta divergence

“…Data-driven automatic speech recognition (ASR) techniques (De Wachter et al, 2003;Aradilla et al, 2005;Deselaers et al, 2007;Sundaram and Bellegarda, 2012;Sainath et al, 2012;Heigold et al, 2012;Sun et al, 2014) became popular in the last decade as a viable alternative after the long dominance of statistical acoustic modeling in the form of the Gaussian mixture models (GMM) in hidden Markov models (HMM) (Bourlard et al, 1996). Templates or exemplars are labeled speech segments of multiple lengths extracted from training data, each associated with a certain class, i.e.…”