MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces

Abstract

Purpose: Current electromagnetic tongue tracking devices are not amenable for daily use and thus not suitable for silent speech interface and other applications. We have recently developed MagTrack, a novel wearable electromagnetic articulograph tongue tracking device. This study aimed to validate MagTrack for potential silent speech interface applications. Method: We conducted two experiments: (a) classification of eight isolated vowels in consonant–vowel–consonant form and (b) continuous silent speech recognition. In these experiments, we used data from healthy adult speakers collected with MagTrack. The performance of vowel classification was measured by accuracies. The continuous silent speech recognition was measured by phoneme error rates. The performance was then compared with results using data collected with commercial electromagnetic articulograph in a prior study. Results: The isolated vowel classification using MagTrack achieved an average accuracy of 89.74% when leveraging all MagTrack signals ( x , y , z coordinates; orientation; and magnetic signals), which outperformed the accuracy using commercial electromagnetic articulograph data (only y , z coordinates) in our previous study. The continuous speech recognition from two subjects using MagTrack achieved phoneme error rates of 73.92% and 66.73%, respectively. The commercial electromagnetic articulograph achieved 64.53% from the same subject (66.73% using MagTrack data). Conclusions: MagTrack showed comparable results with the commercial electromagnetic articulograph when using the same localized information. Adding raw magnetic signals would improve the performance of MagTrack. Our preliminary testing demonstrated the potential for silent speech interface as a lightweight wearable device. This work also lays the foundation to support MagTrack's potential for other applications including visual feedback–based speech therapy and second language learning.

Bioinspired Artificial Intelligence Applications 2023

“…With rapid development of Artificial Intelligence (AI), researchers have found many bioinspired AI applications, such as bioinspired images and speech processing, which can increase accuracy [1][2][3]; bioinspired AI models deployed on edge devices can reduce cost and energy usage [4]; bioinspired AI noise reduction technologies could enhance the bio-signal quality [5][6][7][8]; smart watch-generated bio-signals can be utilized by AI to monitor users' health condition [9,10], and so on.…”