Multilingual Bottleneck Features for Query by Example Spoken Term Detection

Ram, Dhananjay; Miculicich, Lesly; Bourlard, Hervé

doi:10.1109/asru46091.2019.9003752

Cited by 14 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some examples of this approach using different models and units are [26][27][28][80][81][82][83][84][85][86][87][88]. More recently, this approach has been extended with automatic unit discovery [89][90][91] and deep neural networks (DNNs) for extracting bottleneck features [92,93].…”

Section: Query-by-example Spoken Term Detectionmentioning

confidence: 99%

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

et al. 2021

View full text Add to dashboard Cite

The large amount of information stored in audio and video repositories makes search on speech (SoS) a challenging area that is continuously receiving much interest. Within SoS, spoken term detection (STD) aims to retrieve speech data given a text-based representation of a search query (which can include one or more words). On the other hand, query-by-example spoken term detection (QbE STD) aims to retrieve speech data given an acoustic representation of a search query. This is the first paper that presents an internationally open multi-domain evaluation for SoS in Spanish that includes both STD and QbE STD tasks. The evaluation was carefully designed so that several post-evaluation analyses of the main results could be carried out. The evaluation tasks aim to retrieve the speech files that contain the queries, providing their start and end times and a score that reflects how likely the detection within the given time intervals and speech file is. Three different speech databases in Spanish that comprise different domains were employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the SPARL20 database, which contains Spanish parliament sessions. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the evaluation results and some detailed post-evaluation analyses based on specific query properties (in-vocabulary/out-of-vocabulary queries, single-word/multi-word queries and native/foreign queries). The most novel features of the submitted systems are a data augmentation technique for the STD task and an end-to-end system for the QbE STD task. The obtained results suggest that there is clearly room for improvement in the SoS task and that performance is highly sensitive to changes in the data domain.

show abstract

Section: Query-by-example Spoken Term Detectionmentioning

confidence: 99%

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

et al. 2021

View full text Add to dashboard Cite

show abstract

“…As mentioned before, the triplet loss function is designed to learn embeddings that will be spanned closely in the embedding space for similar samples and far apart for non-similar ones. Unlike in previous studies [10], [24] where embeddings were given as fixedlength multidimensional vectors, our embedding model f θ outputs a sequence of embeddings with the length corresponding to the length of the input sequence x. We employ the property of soft-DTW that allows effective calculation of the distance between time-series of different lengths; therefore, the network is designed to generate embedding outputs of varying lengths.…”

Section: Proposed Learning Objectivementioning

confidence: 99%

Learning Acoustic Word Embeddings With Dynamic Time Warping Triplet Networks

et al. 2020

View full text Add to dashboard Cite

In the last years, acoustic word embeddings (AWEs) have gained significant interest in the research community. It applies specifically to the application of acoustic embeddings in the Query-by-Example Spoken Term Detection (QbE-STD) search and related word discrimination tasks. It has been shown that AWEs learned for the word or phone classification in one or several languages can outperform approaches that use dynamic time warping (DTW). In this paper, a new method of learning AWEs in the DTW framework is proposed. It employs a multitask triplet neural network to generate the AWEs. The triplet network learns acoustic representations of words through a comparison of DTW distances. In addition, a multitask objective, including a conventional word classification component, and a triplet loss component is proposed. The triplet loss component applies the DTW distance for the word discrimination task. The multitask objective ensures that the embeddings can be used with DTW directly. Experimental validation shows that the proposed approach is well-suited, but not necessarily restricted to the QbE-STD search. A comparison with several baseline methods shows that the new method leads to a significant improvement of the results on the word discrimination task. An evaluation of the word clustering in the learned embedding space is presented.

show abstract

“…In the first setup, the term is given in a phonetic or grapheme form, and in the second setup, the term is given in an acoustic form. The second setup is referred to as "query-byexample" (QbE) [1,2,3].…”

Section: Introductionmentioning

confidence: 99%

“…More recent works use various forms of deep neural networks for the QbE problem. Some approaches define an audio-embedding space such that if the term was uttered in the input audio example, it would be projected to an embedding vector near the embedding of the audio example [3,7]. Specifically, in [7], they create a shared embedding space for both audio data and the grapheme representation of terms.…”

Section: Introductionmentioning

confidence: 99%

CNN-Based Spoken Term Detection and Localization without Dynamic Programming

Fuchs

Segal

Keshet

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we propose a spoken term detection algorithm for simultaneous prediction and localization of in-vocabulary and out-ofvocabulary terms within an audio segment. The proposed algorithm infers whether a term was uttered within a given speech signal or not by predicting the word embeddings of various parts of the speech signal and comparing them to the word embedding of the desired term. The algorithm utilizes an existing embedding space for this task and does not need to train a task-specific embedding space. At inference the algorithm simultaneously predicts all possible locations of the target term and does not need dynamic programming for optimal search. We evaluate our system on several spoken term detection tasks on read speech corpora.

show abstract

Multilingual Bottleneck Features for Query by Example Spoken Term Detection

Cited by 14 publications

References 26 publications

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

Learning Acoustic Word Embeddings With Dynamic Time Warping Triplet Networks

CNN-Based Spoken Term Detection and Localization without Dynamic Programming

Contact Info

Product

Resources

About