Keyword Spotting for Hearing Assistive Devices Robust to External Speakers

Abstract

Keyword spotting (KWS) is experiencing an upswing due to the pervasiveness of small electronic devices that allow interaction with them via speech. Often, KWS systems are speakerindependent, which means that any person -user or notmight trigger them. For applications like KWS for hearing assistive devices this is unacceptable, as only the user must be allowed to handle them. In this paper we propose KWS for hearing assistive devices that is robust to external speakers. A stateof-the-art deep residual network for small-footprint KWS is regarded as a basis to build upon. By following a multi-task learning scheme, this system is extended to jointly perform KWS and users' own-voice/external speaker detection with a negligible increase in the number of parameters. For experiments, we generate from the Google Speech Commands Dataset a speech corpus emulating hearing aids as a capturing device. Our results show that this multi-task deep residual network is able to achieve a KWS accuracy relative improvement of around 32% with respect to a system that does not deal with external speakers.

Deep Spoken Keyword Spotting: An Overview

“…In this way, Tang and Lin greatly outperformed, with less parameters, standard CNNs [28] in terms of KWS performance, establishing a new state-of-theart back in 2018. Their powerful deep residual architecture so-called res15 has been employed to carry out different KWS studies in areas like robustness for hearing assistive devices [128], [129], filterbank learning [82], and robustness to acoustic noise [130], among others.…”

“…( 4). This approach is preferred over picking classes yielding posteriors above a sensitivity (decision) threshold to be set, since experience tells [82], [128]- [130] that non-streaming deep KWS systems tend to produce very peaked posterior distributions. This might be attributed to the fact that nonstreaming systems do not have to deal with inter-class transition data as in the dynamic case (see the next subsection), but with well-defined, isolated class realizations.…”

“…Therefore, it is obvious that the non-streaming mode lacks some realism from a practical point of view. Despite this, isolated word classification is considered by a number of deep KWS works, e.g., [16], [30], [32], [48]- [52], [58], [69], [82], [89], [99], [109], [125], [128]- [130]. We believe that this is because of the simpler experimental framework with respect to that of the dynamic or streaming case.…”

“…In the above respect, in [128], [129] we studied an alternative way to speaker verification to provide robustness against external speakers (i.e., personalization) in KWS for hearing aids as exemplified by Figure 10. Particularly, we extended the deep residual learning model proposed by Tang and Lin [30] to jointly perform KWS and users' own voice/external speaker detection following a multi-task learning scheme.…”

“…The publicly available Google Speech Commands Dataset [153], [154] has become the de facto open benchmark for [30] for further details). Multiple recent deep KWS works have employed either the first version [16], [30], [32], [43], [48]- [52], [57], [58], [67], [69], [70], [86], [90], [100], [125] or the second version [32], [47], [48], [53], [70], [82], [89], [90], [99], [100], [109], [128]- [130], [159], [175] of the Google Speech Commands Dataset. Despite how valuable this open reference is for KWS research and development, we can raise two relevant points of criticism:…”