Streaming small-footprint keyword spotting using sequence-to-sequence models

Abstract

We develop streaming keyword spotting systems using a recurrent neural network transducer (RNN-T) model: an all-neural, end-toend trained, sequence-to-sequence model which jointly learns acoustic and language model components. Our models are trained to predict either phonemes or graphemes as subword units, thus allowing us to detect arbitrary keyword phrases, without any out-ofvocabulary words. In order to adapt the models to the requirements of keyword spotting, we propose a novel technique which biases the RNN-T system towards a specific keyword of interest.Our systems are compared against a strong sequence-trained, connectionist temporal classification (CTC) based "keyword-filler" baseline, which is augmented with a separate phoneme language model. Overall, our RNN-T system with the proposed biasing technique significantly improves performance over the baseline system.

Query-by-Example On-Device Keyword Spotting

“…Due to the limited data access, direct result comparison with previous works became difficult. Nevertheless, we compared our results with others in Table 2 to show that the results are comparable to that of predefined KWS systems [3,5,4] and query-by-example system [13]. Blanks in the table implies unknown information.…”

“…The query word, 'Hey Snips' is short and false alarms are more likely to occur. The performance is heavily influenced by the type of keyword and this result is also specified in [13].…”

“…Recently, end-to-end NN based query-by-example systems are suggested [13,14]. [13] uses a recurrent neural network transducer (RNN-T) model biased with attention over keyword. [14] suggests to use text query instead of audio.…”

Sub-Band Convolutional Neural Networks for Small-Footprint Spoken Term Classification

“…Amazon Alexa, Google Assistant, Apple Siri), spoken term classification does not have the low-latency constraint since the classification is done at utterance level. Previous works [16,17,18,19] showed that neural networks are very effective in keyword spotting. As tremendous efforts are dedicated into the discovery of effective CNN architectures for further advancing the performance, we argue that it is also important to investigate into effective ways for utilizing computational resource at inference time.…”

Small-Footprint Keyword Spotting with Graph Convolutional Network

“…These methods have demonstrated computational efficiency but failed in capturing local receptive fields and short range context. Various attempts have also been made to build a KWS system with recurrent neural networks (RNNs) [15,16,17,18,19], which is capable of modeling longer temporal context information. However, RNNs may suffer from state saturation while facing continuous input stream, increasing computational cost and detection latency.…”