Low Resource Keyword Search With Synthesized Crosslingual Exemplars

Yusuf, Bolaji; Gundogdu, Batuhan; Saraçlar, Murat

doi:10.1109/taslp.2019.2911164

Cited by 14 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the extended distance metric learning (EDML) network of [22] to learn the document representation X and the query alphabet Q with a distance function defined as…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Evaluation of DTW Subsampling Methods for Keyword Search

Yusuf

Saraçlar

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

State of the art vocabulary-independent spoken term detection methods are typically based on variants of the dynamic time warping (DTW) algorithm since DTW, being based on acoustic sequence matching, allows robust retrieval in settings with scarcity of linguistic resources. However, the DTW comes with a high computational cost which limits its practicality in a deployed server. To this end, we investigate the efficacy of subsampling and propose a neural network architecture to reduce the computational load of DTW-based keyword search. We use a time-subsampled RNN to reduce the frame rate of the document as well as the dimensionality of representation while training it to maintain the cost incurred along the DTW alignment path, thus allowing us to reduce the computational complexity (both space and time) of the search algorithm. Experiments on the Turkish and Zulu limited language packs of the IARPA Babel program show that the proposed methods allow considerable reduction in CPU time (88 times) and memory usage (18 times) without significant loss in search accuracy (0.0270 ATWV). Moreover, even at very high compression levels with lower search precision, high recall rates are maintained, allowing the potential of multi-resolution search.

show abstract

“…We use the extended distance metric learning (EDML) network of [22] to learn the document representation X and the query alphabet Q with a distance function defined as…”

Section: Methodsmentioning

confidence: 99%

“…The multilingual network is then finetuned on each test language and used to obtain posteriorgrams. Using the EDML network described in [22], we obtain 200-dimensional embeddings for DTW-based keyword search from the posteriorgrams.…”

Section: Dataset and Feature Descriptionmentioning

confidence: 99%

An Empirical Evaluation of DTW Subsampling Methods for Keyword Search

Yusuf

Saraçlar

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…This program focused on building fully automatic and noise-robust speech recognition and search systems in a very limited amount of time (e.g., one week) and with limited amount of training data. The languages addressed in that program were low-resourced, such as Cantonese, Pashto, Tagalog, Turkish, Vietnamese, Swahili, Tamil and so on, and significant research has been carried out [13,61,[147][148][149][150][151][152][153][154][155][156][157][158][159].…”

Section: Comparison With Previous Std International Evaluationsmentioning

confidence: 99%

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

et al. 2021

View full text Add to dashboard Cite

The large amount of information stored in audio and video repositories makes search on speech (SoS) a challenging area that is continuously receiving much interest. Within SoS, spoken term detection (STD) aims to retrieve speech data given a text-based representation of a search query (which can include one or more words). On the other hand, query-by-example spoken term detection (QbE STD) aims to retrieve speech data given an acoustic representation of a search query. This is the first paper that presents an internationally open multi-domain evaluation for SoS in Spanish that includes both STD and QbE STD tasks. The evaluation was carefully designed so that several post-evaluation analyses of the main results could be carried out. The evaluation tasks aim to retrieve the speech files that contain the queries, providing their start and end times and a score that reflects how likely the detection within the given time intervals and speech file is. Three different speech databases in Spanish that comprise different domains were employed in the evaluation: the MAVIR database, which comprises a set of talks from workshops; the RTVE database, which includes broadcast news programs; and the SPARL20 database, which contains Spanish parliament sessions. We present the evaluation itself, the three databases, the evaluation metric, the systems submitted to the evaluation, the evaluation results and some detailed post-evaluation analyses based on specific query properties (in-vocabulary/out-of-vocabulary queries, single-word/multi-word queries and native/foreign queries). The most novel features of the submitted systems are a data augmentation technique for the STD task and an end-to-end system for the QbE STD task. The obtained results suggest that there is clearly room for improvement in the SoS task and that performance is highly sensitive to changes in the data domain.

show abstract

“…In the MHA mechanism, the query vector and a set of keyvalue pairs are linearly mapped multiple times respectively [36]- [37]. Each mapping can generate a different attention distribution, and its results are calculated by the Scaled Dot-Product Attention mechanism [35].…”

Section: Multi-head Attention Mechanismmentioning

confidence: 99%

End-to-End Amdo-Tibetan Speech Recognition Based on Knowledge Transfer

Zhu

Huang

2020

IEEE Access

View full text Add to dashboard Cite

The end-to-end speech recognition technology solves the problem that each component is independent and models cannot be jointly optimized in the traditional speech recognition model. It incorporates such components as the acoustic model, language model, and decoding unit of the hybrid model into a single neural network, that can avoid the inherent defects of multiple modules and greatly reduces the complexity of the speech recognition model. In this research, an Amdo-Tibetan speech recognition system is constructed based on Listen, Attend and Spell (LAS) model by the end-to-end speech recognition technology. It can realize the direct conversion from Amdo-Tibetan speech sequence to the corresponding character sequence and greatly reduces the difficulty of building the Amdo-Tibetan speech recognition model. To further improve the performance of the proposed system, the following improvements have been made: firstly, the Multi-Head attention mechanism is introduced to improve the alignment accuracy between state vectors of decoder and encoder; secondly, the label smoothing technique is adopted to solve the problem of over-fitting; thirdly, an N-gram language model is combined with the LAS model to increase the accuracy of speech recognition and the maximum mutual information (MMI) criterion is employed for discriminative training; and finally, transfer learning is utilized to overcome the problem of insufficient training data. Experimental results show that the proposed model can significantly enhance the performance of Amdo-Tibetan speech recognition.

show abstract

Low Resource Keyword Search With Synthesized Crosslingual Exemplars

Cited by 14 publications

References 34 publications

An Empirical Evaluation of DTW Subsampling Methods for Keyword Search

An Empirical Evaluation of DTW Subsampling Methods for Keyword Search

The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses

End-to-End Amdo-Tibetan Speech Recognition Based on Knowledge Transfer

Contact Info

Product

Resources

About