Training Keyword Spotters with Limited and Synthesized Speech Data

Abstract

With the rise of low power speech-enabled devices, there is a growing demand to quickly produce models for recognizing arbitrary sets of keywords. As with many machine learning tasks, one of the most challenging parts in the model creation process is obtaining a sufficient amount of training data. In this paper, we explore the effectiveness of synthesized speech data in training small, spoken term detection models of around 400k parameters. Instead of training such models directly on the audio or low level features such as MFCCs, we use a pre-trained speech embedding model trained to extract useful features for keyword spotting models. Using this speech embedding, we show that a model which detects 10 keywords when trained on only synthetic speech is equivalent to a model trained on over 500 real examples. We also show that a model without our speech embeddings would need to be trained on over 4000 real examples to reach the same accuracy.

Exploring the application of synthetic audio in training keyword spotters

“…Regardless, the application of synthetic data in training low-resource keyword spotter systems has shown promise in recent experiments. Specifically, it was demonstrated that by utilizing a pre-trained speech-embedding model with approximately 400K parameters and weights initialized using human audio data, subsequent training on approximately 2000 synthetic voice examples produced a model with performance only slightly worse than the same model trained with the same number of organic audio examples [4]. The experiments in this paper build on these past works by applying the idea to a new model architecture and training environment.…”

“…One of the most important steps in ensuring a frictionless experience for customers of voice assistants is "waking" the device up for interaction when the customer intends to use the device, and, importantly, not "waking" when the customer does not. This is accomplished with specialized "wakeword models" that detect keywords spoken by users and initiate interactions with the device [2,3,4].…”

Wav2KWS: Transfer Learning From Speech Representations for Keyword Spotting

“…A head model has shown the benefits of embedding, which is built on learning many short utterances [23]. The head model quickly converges to the model from the shared weights of pretrained embeddings.…”

“…Consequently, neural network optimization becomes difficult given the scarce training data available. To overcome data scarcity for KWS, a pretrained head model and synthesized data have been used [23]. Lin et al [23] state that building a state-of-the-art (SOTA) KWS model requires more than 4000 recorded human speech samples per command.…”

Progressive Voice Trigger Detection: Accuracy vs Latency

“…The presence of the trigger phrase at the beginning of an utterance helps distinguish audio that is directed towards the assistant from background speech. The problem of accurately detecting a trigger phrase is known as voice trigger detection [1,2], wake-word detection [3,4], or keyword spotting [5,6,7,8,9]. This work is motivated by the observation that audio following the trigger phrase can contain a strong signal about whether an utterance was directed towards the assistant or not.…”