Resource configurable spoken query detection using Deep Boltzmann Machines

Zhang, Yaodong; Salakhutdinov, Ruslan; Chang, Hung-An; Glass, James

doi:10.1109/icassp.2012.6289082

Cited by 49 publications

(53 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the case where we use none of the annotations, we train a 61-mixture GMM Fig. 2: System performance with respect to different percentage of annotations used for back propagation first and then assign the labels according to the index of the mixture with the highest posterior probability [7].…”

Section: Resultsmentioning

confidence: 99%

“…As Zhang et. al [7] have reported that using DBN posteriorgrams greatly improves the performance on the task of keyword spotting, which is essentially a comparison task, we would also like to explore the use of DBN posteriorgrams in our system, with various settings for training.…”

Section: Related Workmentioning

confidence: 99%

“…One of the attractive characteristics of DBNs is that the pre-training step does not require any annotation of the data, while the back-propagation step allows us to fine-tune the pre-trained generative model with some labels. In other words, the model can be trained in a semi-supervised fashion [5,7]. One of the challenges we have encountered in our prior work is that some mixtures in the GMM capture the difference in speakers rather than phonetic units, resulting in some parts of the mis-alignment not necessarily corresponding to mispronunciation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mispronunciation detection via dynamic time warping on deep belief network-based posteriorgrams

Lee

Zhang

Glass

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

Self Cite

View full text Add to dashboard Cite

In this paper, we explore the use of deep belief network (DBN) posteriorgrams as input to our previously proposed comparison-based system for detecting word-level mispronunciation. The system works by aligning a nonnative utterance with at least one native utterance and extracting features that describe the degree of mis-alignment from the aligned path and the distance matrix. We report system performance under different DBN training scenarios: pre-training and fine-tuning with either native data only or both native and nonnative data. Experimental results have shown that by substituting the system input from MFCC or Gaussian posteriorgrams obtained in a fully unsupervised manner to DBN posteriorgrams, the system performance can be improved by at least 10.4% relatively. Moreover, the system performance remains steady when only 30% of the annotations being used.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mispronunciation detection via dynamic time warping on deep belief network-based posteriorgrams

Lee

Zhang

Glass

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Parallelizing the model estimation procedure on a multicore system or a graphics processor unit using techniques similar to [22] would allow for larger audio corpora to be analyzed. The use of new unsupervised and semi-supervised acoustic models [17,24] may prove to be useful for improving the performance of the spoken term discovery procedure. Finally, reworking the models presented in this paper into a fully Bayesian framework would expand the flexibility of the models and allow their size to automatically scale with the amount of data used.…”

Section: Discussionmentioning

confidence: 99%

Zero resource spoken audio corpus analysis

Harwath

Hazen

Glass³

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

Zero-resource speech processing involves the automatic analysis of a collection of speech data in a completely unsupervised fashion without the benefit of any transcriptions or annotations of the data. In this paper, our zero-resource system seeks to automatically discover important words, phrases and topical themes present in an audio corpus. This system employs a segmental dynamic time warping (S-DTW) algorithm for acoustic pattern discovery in conjunction with a probabilistic model which treats the topic and pseudo-word identity of each discovered pattern as hidden variables. By applying an Expectation-Maximization (EM) algorithm, our system estimates the latent probability distributions over the pseudo-words and topics associated with the discovered patterns. Using this information, we produce acoustic summaries of the dominant topical themes of the audio document collection.

show abstract

“…Due to distribution learning capability of GaussianBernoulli Restricted Boltzmann Machines (GBRBM), RBMbased posteriorgrams were found to be comparable to Gaussian posteriorgrams. Restricted Boltzmann Machines (RBM) and Deep Belief Network (DBN) were used for QbE-STD tasks as an alternative to Gaussian posteriorgrams [9], [10].…”

Section: Introductionmentioning

confidence: 99%

VTLN-warped Gaussian posteriorgram for QbE-STD

Madhavi

Patil

2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-Vocal Tract Length Normalization (VTLN) is a very important speaker normalization technique for speech recognition tasks. In this paper, we propose the use of Gaussian posteriorgram of VTLN-warped spectral features for a Queryby-Example Spoken Term Detection (QbE-STD). This paper presents the use of a Gaussian Mixture Model (GMM) framework for estimation of VTLN warping factor. This GMM framework does not require phoneme-level transcription and hence, it can be useful for unsupervised tasks. We propose the iterative approach for VTLN warping factor estimation with two GMM training approaches, namely, Expectation-Maximization (EM) and Deterministic Annealing-Expectation Maximization (DAEM). The VTLN-warped Gaussian posteriorgram gave the better QbE-STD performance. The performance of TIMIT QbE-STD was investigated with different evaluation factors, such as a number of Gaussian components in GMM, various local constraints, and a number of iterations in VTLN warping factor estimation. VTLNwarped Gaussian posteriorgram reduces the speaker-specific variation in Gaussian posteriorgram and hence, it is expected to give better performance than Gaussian posteriorgram.

show abstract

Resource configurable spoken query detection using Deep Boltzmann Machines

Cited by 49 publications

References 8 publications

Mispronunciation detection via dynamic time warping on deep belief network-based posteriorgrams

Mispronunciation detection via dynamic time warping on deep belief network-based posteriorgrams

Zero resource spoken audio corpus analysis

VTLN-warped Gaussian posteriorgram for QbE-STD

Contact Info

Product

Resources

About