Tree-structured vector quantization for speech recognition

Barszcz, M.; Chen, W.; Kenny, Patrick

doi:10.1006/csla.2000.0143

Cited by 7 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 Let t = 1 and R = ∅. 6 / * The process of GLA * / 7 Partition the remaining input vector X − R into a set of disjoint blocks B j , j = 1, 2, . .…”

Section: A Notationsmentioning

confidence: 99%

“…In general, VQ uses a codebook to encode and decode the signal and transmits the compressed signal over a communication channel. Because it is simple and easy to implement, VQ has been widely used in different applications such as pattern recognition [1], [2], [3], [4], pattern compression [5], [6], speech recognition [7], [8], and face detection [9]. Also called Generalized Lloyd Algorithm (GLA), Linde-Buzo-Gary (LBG) algorithm [10], due to its simplicity and relatively good fidelity, is the most widely used VQ method.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast VQ codebook generation via Pattern Reduction

Lee

Tsai

Chiang

et al. 2008

2008 IEEE International Conference on Systems, Man and Cybernetics

View full text Add to dashboard Cite

In this paper, we present a simple but fast codebook generation algorithm, called PREGLA (Pattern Reduction Enhanced GLA). The proposed algorithm is fundamentally different from the previous approaches in that the latter focuses on reducing the size of the codebook whereas the former focuses on using pattern reduction to reduce the computation time. The proposed algorithm is motivated by the observation that input vectors that are "static" during the training process can be considered as part of the final solutions and thus can be compressed and removed to eliminate the redundant computations at the later iterations of the training process. To evaluate the performance of the proposed algorithm, we compare the proposed algorithm with "efficient" VQ algorithms published recently such as Nearest Partition Set Search, Fast Vector Quantization Algorithm, and standard GLA. Our experimental results indicate that the proposed algorithm can reduce the computation time from 27.86% up to about 80.45% compared to that of standard GLA and other fast GLA-based algorithms alone.

show abstract

“…5 Let t = 1 and R = ∅. 6 / * The process of GLA * / 7 Partition the remaining input vector X − R into a set of disjoint blocks B j , j = 1, 2, . .…”

Section: A Notationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fast VQ codebook generation via Pattern Reduction

Lee

Tsai

Chiang

et al. 2008

2008 IEEE International Conference on Systems, Man and Cybernetics

View full text Add to dashboard Cite

show abstract

“…This is probably because DHMM was most popular in 1980's, when the memory resources were quite limited, and couldnot afford large models. The only few works with large codebooks that we are aware of are in [5,53,24].…”

Section: Very Large Codebook Dhmmmentioning

confidence: 99%

“…In [5], both VQ and MMI based tree-structured codebook were investigated for a very large codebook system with 16∼64 K codewords. To robustly estimate the model parameters, three types of tree-structured smoothing techniques (mixture smoothing, smoothing by adding 1, and Gaussian smoothing) were explored.…”

Section: Very Large Codebook Dhmmmentioning

confidence: 99%

“…It was found that different smoothing methods yielded comparable performances, and MMI-based treestructured codebook did not seem to be better than VQ-based ones in such a large codebook system. Unfortunately, limited by the resources, the evaluation set used in [5] is not a standard one, and the very CDHMM baseline is not strong.…”

Section: Very Large Codebook Dhmmmentioning

confidence: 99%

See 1 more Smart Citation

The use of discrete distributions with a very large codebook for automatic speech recognition and speaker verification

Mak

View full text Add to dashboard Cite

Chapter 1 Introduction 1 1.1 Problems with Very Large Codebook Discrete Systems 3 1.2 Contributions of the Thesis 4 1.3 Thesis Outline 6 Chapter 2 Introduction to Automatic Speech Recognition (ASR) 8 2.1 Definition of ASR 8 2.1.1 Statistical ASR 9 2.1.2 ASR System Performance Evaluation Criterion 2.2 Hidden Markov Model (HMM) for ASR 2.2.1 Dynamic Features for HMM 2.2.2 Training of and Recognition with HMM in ASR 2.3 Different Types of HMM in ASR 2.3.1 Continuous Density HMM (CDHMM) 2.3.2 Semi-continuous HMM (SCHMM) 2.3.3 Discrete HMM (DHMM) 2.3.4 Multiple-stream HMM v Chapter 3 Related Work in DHMM 3.1 Different Types of Codebooks 3.1.1 Codebook by Unsupervised Construction 3.1.2 Codebook by Supervised Construction LIST OF FIGURES 2.1 Phone HMM /i/: first-order 3-state left-to-right topology 2.2 Word HMM "it": constructed from phone HMMs /i/ and /t/ 2.3 Different types of HMM 2.4 Examples of 2-stream HMM systems 3.1 Codebooks produced by unsupervised and supervised construction. Each method partitions the acoustic space into 2 parts. The dots represent training samples in codebook construction, with colors representing different classes. 4.1 SVQ codebooks, where x t is a d-dimensional feature vector at frame t, and VQ(x t) is its corresponding full space VQ codeword constructed from SVQ or SQ codewords. Notice that when L = d, it becomes SQ codebooks. 4.2 An example of a 2-stream system with SVQ codebooks. Each stream is further split into 2 subvectors with 2 SVQ codebooks. 4.3 Relationship between model size and recognition error rate for HD-DHMM with SQ codebooks 4.4 One-stream SHDDHMM architecture overview. The shaded ellipse represents the global pool of bases, spanning the subspace. Each state output discrete distribution table (i.e., b i , and b j), lies in the subspace. The state-dependent weights (the dotted lines) and the global pool of bases (the shaded ellipse) are temporary parameters, which only exist during model training; the final model only stores b i and b j. 4.5 An example of a 2-stream SHDDHMM, each stream is further split into 2 subvectors. A global pool of bases is stored for each stream independently, and the number of bases in a pool is 3. 4.6 Stream weight estimation results for SHDDHMM with iterative linear programming 4.7 Operating characteristics of various SI-84 models (finding codeword time is included for discrete models.) 4.8 An example of "smoothing by adding 1" technique 5.1 An example of computer access control using speaker verification technology 5.2 GMM-UBM based speaker verification system 5.3 SV operating characteristics of various models, when the UBM and speaker models are pre-loaded to the memory. 5.4 SV operating characteristics of various models, when the UBM and speaker models are loaded on-the-fly. ix LIST OF TABLES 4.1 Experimental settings for different ASR systems 29 4.2 Bit allocation for each stream for HDDHMM in [53] 38 4.3 Baseline model performance on WSJ SI-84 42 4.4 Effect of codebook size for 4s-HDDHMM with SQ codebooks 44 4.5 Comparison of different inte...

show abstract