Subspace Detection of DNN Posterior Probabilities via Sparse Representation for Query by Example Spoken Term Detection

Ram, Dhananjay; Asaei, Afsaneh; Bourlard, Hervé

doi:10.21437/interspeech.2016-1278

Cited by 10 publications

(7 citation statements)

References 21 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This low-dimensional structure is the result of the constrained articulatory mechanism of human speech production [11], [12], which leads to the generation of linguistic units (e.g., phones, senones) lying on non-linear manifolds. As already shown in [13], [14], [15], these manifolds can be modeled as a union of low-dimensional subspaces and sparse representation is found to be a promising technique to model these subspaces. It is the goal of the present work to investigate how this sparsity property can be exploited to further improve state-of-the-art QbE-STD system.…”

Section: Introductionmentioning

confidence: 88%

“…In the context of speech processing, sparse recovery has already been studied for robust speech recognition [18], [19], [20], enhanced acoustic modeling [14], [15] as well as spoken query detection [13], [21], [22]. In our earlier work [13], we cast the query detection problem as subspace detection between query and non-query speech where the corresponding subspaces are modeled through dictionary learning for sparse representation. Given these dictionaries, detection of each frame is performed based on the ratio of the two corresponding sparse representation reconstruction errors, and the frame-level decisions are accumulated by counting the continuous number of frames detected as the query.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sparse Subspace Modeling for Query by Example Spoken Term Detection

Ram

Asaei

Bourlard

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Abstract-This paper focuses on the problem of query by example spoken term detection (QbE-STD) in zero-resource scenario. Current state-of-the-art approaches to tackle this problem rely on dynamic programming based template matching techniques using phone posterior features extracted at the output of a deep neural network (DNN). Previously, it has been shown that the space of phone posteriors is highly structured, as a union of low-dimensional subspaces. To exploit the temporal and sparse structure of the speech data, we investigate here three different QbE-STD systems based on sparse model recovery. More specifically, we use query examples to model the query subspace using dictionary for sparse coding. Reconstruction errors calculated using sparse representation of feature vectors are then used to characterize the underlying subspaces. The first approach uses these reconstruction errors in a dynamic programming framework to detect the spoken query, resulting in a much faster search compared to standard template matching. The other two methods aim at merging template matching and sparsity based approaches to further improve the performance. The first one proposes to regularize the template matching local distances using sparse reconstruction errors. The second approach aims at using the sparse reconstruction errors to rescore (improve) the template matching likelihood. Experiments on two different databases (AMI and MediaEval) show that the proposed hybrid systems perform better than a highly competitive QbE-STD baseline system.

show abstract

Section: Introductionmentioning

confidence: 88%

Section: Introductionmentioning

confidence: 99%

Sparse Subspace Modeling for Query by Example Spoken Term Detection

Ram

Asaei

Bourlard

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Exploiting this property enables a hierarchical speech classification and recognition framework based on structured sparse modeling of posterior exemplars [9,11]. In addition, the low-dimensional subspaces can be modeled through dictionary learning for sparse coding to enable unsupervised adaptation and enhanced acoustic modeling for speech recognition [10,12].…”

Section: State-of-the-art Solutions and Challengesmentioning

confidence: 99%

“…In addition, the low-dimensional subspaces can be modeled through dictionary learning for sparse coding to enable unsupervised adaptation and enhanced acoustic modeling for speech recognition [10,12]. Sparse subspace modeling of the posterior exemplars are also found promising for query-by-example spoken term detection (QbE-STD) [7,11,13].…”

Section: State-of-the-art Solutions and Challengesmentioning

confidence: 99%

Phonological Posterior Hashing for Query by Example Spoken Term Detection

Asaei¹,

Ram²,

Bourlard³

2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

State of the art query by example spoken term detection (QbE-STD) systems in zero-resource conditions rely on representation of speech in terms of sequences of class-conditional posterior probabilities estimated by deep neural network (DNN). The posteriors are often used for pattern matching or dynamic time warping (DTW). Exploiting posterior probabilities as speech representation propounds diverse advantages in a classification system. One key property of the posterior representations is that they admit a highly effective hashing strategy that enables indexing a large audio archive in divisions for reducing the search complexity. Moreover, posterior indexing leads to a compressed representation and enables pronunciation dewarping and partial detection with no need for DTW. We exploit these characteristics of the posterior space in the context of redundant hash addressing for query-by-example spoken term detection (QbE-STD). We evaluate the QbE-STD system on AMI corpus and demonstrate that tremendous speedup and superior accuracy is achieved compared to the state-of-the-art pattern matching solution based on DTW. The system has the potential to enable massively large scale spoken query detection.

show abstract

“…Exploiting this property enables hierarchical speech recognition and classification frameworks based on sparse modeling of phonetic posterior exemplars [10,11]. Furthermore, the low-dimensional subspaces can be modeled through dictionary learning for sparse representation, and projection of the posteriors into the space characterized over the training data reduces the mismatch of the testing posteriors, and leads to enhanced acoustic modeling for speech recognition [8,9].…”

Section: Introductionmentioning

confidence: 99%

Phonetic and Phonological Posterior Search Space Hashing Exploiting Class-Specific Sparsity Structures

Asaei¹,

Luyet²,

Cerňak³

et al. 2016

Interspeech 2016

Self Cite

View full text Add to dashboard Cite

This paper shows that exemplar-based speech processing using class-conditional posterior probabilities admits a highly effective search strategy relying on posteriors' intrinsic sparsity structures. The posterior probabilities are estimated for phonetic and phonological classes using deep neural network (DNN) computational framework. Exploiting the class-specific sparsity leads to a simple quantized posterior hashing procedure to reduce the search space of posterior exemplars. To that end, small number of quantized posteriors are regarded as representatives of the posterior space and used as hash keys to index subsets of neighboring exemplars. The k nearest neighbor (kNN) method is applied for posterior based classification problems. The phonetic posterior probabilities are used as exemplars for phonetic classification whereas the phonological posteriors are used as exemplars for automatic prosodic event detection. Experimental results demonstrate that posterior hashing improves the efficiency of kNN classification drastically. This work encourages the use of posteriors as discriminative exemplars appropriate for large scale speech classification tasks.

show abstract

Subspace Detection of DNN Posterior Probabilities via Sparse Representation for Query by Example Spoken Term Detection

Cited by 10 publications

References 21 publications

Sparse Subspace Modeling for Query by Example Spoken Term Detection

Sparse Subspace Modeling for Query by Example Spoken Term Detection

Phonological Posterior Hashing for Query by Example Spoken Term Detection

Phonetic and Phonological Posterior Search Space Hashing Exploiting Class-Specific Sparsity Structures

Contact Info

Product

Resources

About