Query-by-Example On-Device Keyword Spotting

Kim, Byeonggeun; Lee, Mingu; Lee, Jin-Kyu; Kim, Yeon-Seok; Hwang, Kyuwoong

doi:10.1109/asru46091.2019.9004014

Cited by 20 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, we follow the training details of CIFAR experiments in [17]. Keyword Spotting For keyword spotting, we use Qualcomm Keyword Speech Dataset 3 [20] and ResNet-26 [21] for the baseline network. Qualcomm keyword dataset contains 4,270 utterances of four English keywords spoken by 42-50 people.…”

Section: Methodsmentioning

confidence: 99%

Prototype-Based Personalized Pruning

Kim

Chang²,

Yun³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Nowadays, as edge devices such as smartphones become prevalent, there are increasing demands for personalized services. However, traditional personalization methods are not suitable for edge devices because retraining or finetuning is needed with limited personal data. Also, a full model might be too heavy for edge devices with limited resources. Unfortunately, model compression methods which can handle the model complexity issue also require the retraining phase. These multiple training phases generally need huge computational cost during on-device learning which can be a burden to edge devices. In this work, we propose a dynamic personalization method called prototype-based personalized pruning (PPP). PPP considers both ends of personalization and model efficiency. After training a network, PPP can easily prune the network with a prototype representing the characteristics of personal data and it performs well without retraining or finetuning. We verify the usefulness of PPP on a couple of tasks in computer vision and Keyword spotting.

show abstract

Section: Methodsmentioning

confidence: 99%

Prototype-Based Personalized Pruning

Kim

Chang²,

Yun³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…For example, this is the case for the speech corpora reported in [26], [122] and [9], which were collected, respectively, from Mobvoi's TicKasa Fox, Google's Google Home and Xiaomi's AI Speaker smart speakers. Unfortunately, only seven out of twenty six datasets in Table 1 are publicly available: one from Sonos [169], two different arrangements of AISHELL-2 [199] (used in [98]), the Google Speech Commands Dataset v1 [153] and v2 [154], the Hey Snapdragon Keyword Dataset [200], and Hey Snips [78], [198] (also used in, e.g., [53], [177]). In case of interest in getting access to any of these speech corpora, the reader is pointed to the corresponding references indicated in Table 1.…”

Section: Datasetsmentioning

confidence: 99%

“…This is for example the case for voice activation of voice assistants, where privacy is a major concern [232] since this application involves streaming voice to a cloud server. As a result, a popular variant of the ROC and DET curves is that one replacing false positive rate along the xaxis by the number of false alarms per hour [8], [28], [31], [59], [60], [156], [162], [200]. By this, a practitioner can just set a very small number of false alarms per hour (e.g., 1) and identify the system with the highest (lowest) true positive (false negative) rate for deployment.…”

Section: B Receiver Operating Characteristic and Detection Error Trade-off Curvesmentioning

confidence: 99%

Deep Spoken Keyword Spotting: An Overview

López-Espejo¹,

Hansen²,

Jensen³

2022

IEEE Access

View full text Add to dashboard Cite

Spoken keyword spotting (KWS) deals with the identification of keywords in audio streams and has become a fast-growing technology thanks to the paradigm shift introduced by deep learning a few years ago. This has allowed the rapid embedding of deep KWS in a myriad of small electronic devices with different purposes like the activation of voice assistants. Prospects suggest a sustained growth in terms of social use of this technology. Thus, it is not surprising that deep KWS has become a hot research topic among speech scientists, who constantly look for KWS performance improvement and computational complexity reduction. This context motivates this paper, in which we conduct a literature review into deep spoken KWS to assist practitioners and researchers who are interested in this technology. Specifically, this overview has a comprehensive nature by covering a thorough analysis of deep KWS systems (which includes speech features, acoustic modeling and posterior handling), robustness methods, applications, datasets, evaluation metrics, performance of deep KWS systems and audio-visual KWS. The analysis performed in this paper allows us to identify a number of directions for future research, including directions adopted from automatic speech recognition research and directions that are unique to the problem of spoken KWS.INDEX TERMS Keyword spotting, deep learning, acoustic model, small footprint, robustness.

show abstract

“…MHAtt-RNN [20] utilizes multi-head attention over it. On the other hand, there are automatic speech recognition based approaches [21,22]. Note that the approaches are successful but typically are not efficient in terms of the number of parameters compared to CNN-based approaches.…”

Section: Related Workmentioning

confidence: 99%

Broadcasted Residual Learning for Efficient Keyword Spotting

Kim¹,

Chang²,

Lee³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Keyword spotting is an important research field because it plays a key role in device wake-up and user interaction on smart devices. However, it is challenging to minimize errors while operating efficiently in devices with limited resources such as mobile phones. We present a broadcasted residual learning method to achieve high accuracy with small model size and computational load. Our method configures most of the residual functions as 1D temporal convolution while still allows 2D convolution together using a broadcasted-residual connection that expands temporal output to frequency-temporal dimension. This residual mapping enables the network to effectively represent useful audio features with much less computation than conventional convolutional neural networks. We also propose a novel network architecture, Broadcasting-residual network (BC-ResNet), based on broadcasted residual learning and describe how to scale up the model according to the target device's resources. BC-ResNets achieve state-of-the-art 98.0% and 98.7% top-1 accuracy on Google speech command datasets v1 and v2, respectively, and consistently outperform previous approaches, using fewer computations and parameters.

show abstract

Query-by-Example On-Device Keyword Spotting

Cited by 20 publications

References 26 publications

Prototype-Based Personalized Pruning

Prototype-Based Personalized Pruning

Deep Spoken Keyword Spotting: An Overview

Broadcasted Residual Learning for Efficient Keyword Spotting

Contact Info

Product

Resources

About