Preliminary Evaluation of a Silent Speech Interface based on Intra-Oral Magnetic Sensing

Abstract

This paper addresses the hardware challenges faced in developing a practical silent speech interface (SSI) for post-laryngectomy speech rehabilitation. Although a number of SSIs have been developed, many are still deemed as impractical due to a high degree of intrusiveness and discomfort, hence limiting their transition to outside of the laboratory environment. The aim of this paper is to build upon our previous work, in developing a user-centric prototype and enhancing its desirable features. A new Permanent Magnet Articulography (PMA) system is presented which fits within the palatal cavity of the user's mouth, giving unobtrusive appearance and high portability. The prototype is comprised of a miniaturised circuit constructed using commercial off-the-shelf (COTS) components and is implemented in the form of a dental retainer, which is mounted under roof of the user's mouth and firmly clasps onto the upper teeth. Preliminary evaluation via speech recognition experiments demonstrates that the intraoral prototype achieves word recognition accuracy of 75.7%, slightly lower than its predecessor. Nonetheless, the intraoral design is expected to improve the stability and robustness of the PMA system with a much improved appearance since it can be completely hidden inside the user's mouth.

Silent Speech Interfaces for Speech Restoration: A Review

“…In contrast, SSI-based speech restoration promises to overcome many of the above issues. Thus, the possibility of recognising speech from speech-related biosignals has been demonstrated for a variety of sensing techniques, such as sEMG [28], [30], [79], [86]- [88], electromagnetic articulography (EMA) [89], [90], permanent magnetic articulography (PMA) [24], [26], [91]- [93], and imaging technologies based on video and ultrasound [20], [94], [95]. Furthermore, direct speech generation from the captured biosignals (see Section III-B) is another possibility, having this approach the potential to restore the person's own voice, if enough recordings of the pre-laryngectomy voice are available for training [96].…”

“…First, while many techniques are designed to allow some portability and to be generally non-invasive, some problems remain. The equipment is not discreet and/or comfortable enough to be used as a wearable in real-world practice [26], [278] and may be insufficiently robust against sensor misalignment [279], [280]. Second, the linguistic information captured by these devices is often limited.…”

Silent Speech: Restoring the Power of Speech to People whose Larynx has been Removed

“…techniques (typically deep neural networks [13]). For capturing articulator movement we use Permanent Magnet Articulography (PMA) [14,15,16,17], a technology developed by our collaborators at the University of Hull in which small magnets are attached to the lips and tongue and the magnetic field generated when the articulators move is captured by sensors close to the mouth (see Fig. 1 for a picture of the PMA system).…”

Voice Restoration After Laryngectomy Based on Magnetic Sensing of Articulator Movement and Statistical Articulation-to-Speech Conversion

“…As shown in Fig. 1, an external headset is used to hold the four magnetic sensors employed in the current prototype, though other arrangements such as an intra-oral device similar to a dental retainer has been and are currently being investigated [5,4]. From the four sensors in Fig.…”