Deep Spoken Keyword Spotting: An Overview

Abstract

Spoken keyword spotting (KWS) deals with the identification of keywords in audio streams and has become a fast-growing technology thanks to the paradigm shift introduced by deep learning a few years ago. This has allowed the rapid embedding of deep KWS in a myriad of small electronic devices with different purposes like the activation of voice assistants. Prospects suggest a sustained growth in terms of social use of this technology. Thus, it is not surprising that deep KWS has become a hot research topic among speech scientists, who constantly look for KWS performance improvement and computational complexity reduction. This context motivates this paper, in which we conduct a literature review into deep spoken KWS to assist practitioners and researchers who are interested in this technology. Specifically, this overview has a comprehensive nature by covering a thorough analysis of deep KWS systems (which includes speech features, acoustic modeling and posterior handling), robustness methods, applications, datasets, evaluation metrics, performance of deep KWS systems and audio-visual KWS. The analysis performed in this paper allows us to identify a number of directions for future research, including directions adopted from automatic speech recognition research and directions that are unique to the problem of spoken KWS.INDEX TERMS Keyword spotting, deep learning, acoustic model, small footprint, robustness.

AB/BA analysis: A framework for estimating keyword spotting recall improvement while maintaining audio privacy

“…As shown in Figure 1 (B), the data collected by model A is used by model B for keyword spotting, and model B is run on the data collected by the model A. A detailed review on keyword spotting can be found in (López-Espejo et al, 2021). Running a keyword spotter on an utterance i will produce a score s i representing how likely it is to have detected the keyword.…”

Deep Learning-Based Detection of Inappropriate Speech Content for Film Censorship

“…The normal class consist of 5100 original and 45900 augmented samples using various augmentation techniques that were detailed in [66]. However, for this work only 12100 samples of normal class were used to mitigate the issue of imbalance dataset and foul language data samples scarcity as data imbalance and rarity is a major issue for KWS systems [9]. Additionally, the effect of data augmentation has led to improving model's performance and robustness to noise [66].…”

“…Spoken keyword spotting (KWS) is a fast-growing technology due to the increased usage often coupled with deep learning techniques that involves the identification of keywords in audio streams [9]. As a consequence of the rapid growth of human-machine interaction via voice, the social usage of this technology is expected to achieve sustainable growth.…”

“…Deep spoken keyword spotting system [30][31][32] typically contains three main blocks [9]: 1) the speech feature extractor that converts the input signal to a compact speech representation, 2) the deep learning-based acoustic model that generates posteriors over the keyword and filler (nonkeyword) classes based on the speech features, and 3) the posterior handler that processes the temporal sequence of posteriors to determine the possible existence of keywords in the input signal.…”

“…Mel-scale-related features, low-precision features, learnable filter-bank features, and other features are the most relevant speech features used in deep KWS systems [9]. Speech features based on the perceptually-motivated Mel-scale filter-bank, e.g., log-Mel spectral coefficients and Mel-frequency cepstral coefficients (MFCCs), have been commonly utilized in the areas of ASR and KWS.…”