Spoken term detection with Connectionist Temporal Classification: A novel hybrid CTC-DBN decoder

Wöllmer, Martin; Eyben, Florian; Schuller, Björn; Rigoll, Gerhard

doi:10.1109/icassp.2010.5494980

Cited by 11 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a second baseline model, we evaluated the keyword spotting performance of a hybrid BLSTM-HMM system, since this approach was shown to prevail over the standard HMM approach [61]. Unlike the proposed Tandem model, the hybrid approach exclusively uses BLSTM phoneme predictions for keyword detection.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

et al. 2010

Self Cite

View full text Add to dashboard Cite

Robustly detecting keywords in human speech is an important precondition for cognitive systems, which aim at intelligently interacting with users. Conventional techniques for keyword spotting usually show good performance when evaluated on well articulated read speech. However, modeling natural, spontaneous, and emotionally colored speech is challenging for today's speech recognition systems and thus requires novel approaches with enhanced robustness. In this article, we propose a new architecture for vocabulary independent keyword detection as needed for cognitive virtual agents such as the SEMAINE system. Our word spotting model is composed of a Dynamic Bayesian Network (DBN) and a bidirectional Long Short-Term Memory (BLSTM) recurrent neural net. The BLSTM network uses a self-learned amount of contextual information to provide a discrete phoneme prediction feature for the DBN, which is able to distinguish between keywords and arbitrary speech. We evaluate our Tandem BLSTM-DBN technique on both read speech and spontaneous emotional speech and show that our method significantly outperforms conventional Hidden Markov Model-based approaches for both application scenarios.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Typical phoneme prediction errors made by the CTC network are modeled by the HMM layer of the hybrid system (similar to the trained CPFs p(b t |s t ) for the Tandem model). For further details on the hybrid approach, the reader is referred to [61].…”

Section: Methodsmentioning

confidence: 99%

Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

et al. 2010

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [22], a keyword spotter for specific keywords using BLSTM-CTC is first proposed but only the most intuitive postprocessing is applied. Besides, [23] comes up with a complicated CTC-DBN decoder which is shown to outperform a Keyword-Filler Hidden Markov Model system. In this paper, the LSTM is unidirectional so the number of parameters is reasonable for many applications.…”

Section: Introductionmentioning

confidence: 99%

Unrestricted Vocabulary Keyword Spotting Using LSTM-CTC

Zhuang

Chang

et al. 2016

Interspeech 2016

View full text Add to dashboard Cite

Keyword spotting (KWS) aims to detect predefined keywords in continuous speech. Recently, direct deep learning approaches have been used for KWS and achieved great success. However, these approaches mostly assume fixed keyword vocabulary and require significant retraining efforts if new keywords are to be detected. For unrestricted vocabulary, HMM based keywordfiller framework is still the mainstream technique. In this paper, a novel deep learning approach is proposed for unrestricted vocabulary KWS based on Connectionist Temporal Classification (CTC) with Long Short-Term Memory (LSTM). Here, an LSTM is trained to discriminant phones with the CTC criterion. During KWS, an arbitrary keyword can be specified and it is represented by one or more phone sequences. Due to the property of peaky phone posteriors of CTC, the LSTM can produce a phone lattice. Then, a fast substring matching algorithm based on minimum edit distance is used to search the keyword phone sequence on the phone lattice. The approach is highly efficient and vocabulary independent. Experiments showed that the proposed approach can achieve significantly better results compared to a DNN-HMM based keyword-filler decoding system. In addition, the proposed approach is also more efficient than the DNN-HMM KWS baseline.

show abstract

“…Future experiments will include the design of bottle-neck [5] BLSTM networks as well as the combination of multi-stream BLSTM-HMM systems with techniques for feature enhancement in order to allow further performance gains in noisy conditions. A further promising aspect for future research is to use the principle of connectionist temporal classification [6] for continuous speech recognition.…”

Section: Discussionmentioning

confidence: 99%

“…Most studies concentrate on improving the front-or back-end of ASR systems based on Hidden Markov Models (HMM), however, strategies towards improving ASR in challenging conditions by combining the HMM principle with multilayer perceptrons (MLP) or recurrent neural networks (RNN) are gaining more and more attention [4,5,6]. These techniques can be roughly categorized into hybrid approaches that apply neural networks to generate state posteriors for HMMs, and Tandem approaches that use the network output as features instead of (or in combination with) standard cepstral features.…”

Section: Introductionmentioning

confidence: 99%

A multi-stream ASR framework for BLSTM modeling of conversational speech

Wöllmer

Eyben

Schuller

et al. 2011

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We propose a novel multi-stream framework for continuous conversational speech recognition which employs bidirectional Long Short-Term Memory (BLSTM) networks for phoneme prediction. The BLSTM architecture allows recurrent neural nets to model longrange context, which led to improved ASR performance when combined with conventional triphone modeling in a Tandem system. In this paper, we extend the principle of joint BLSTM and triphone modeling to a multi-stream system which uses MFCC features and BLSTM predictions as observations originating from two independent data streams. Using the COSINE database, we show that this technique prevails over a recently proposed single-stream Tandem system as well as over a conventional HMM recognizer.

show abstract

Spoken term detection with Connectionist Temporal Classification: A novel hybrid CTC-DBN decoder

Cited by 11 publications

References 10 publications

Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

Unrestricted Vocabulary Keyword Spotting Using LSTM-CTC

A multi-stream ASR framework for BLSTM modeling of conversational speech

Contact Info

Product

Resources

About