Towards an Intraoral-Based Silent Speech Restoration System for Post-laryngectomy Voice Replacement

Abstract

Abstract. Silent Speech Interfaces (SSIs) are alternative assistive speech technologies that are capable of restoring speech communication for those individuals who have lost their voice due to laryngectomy or diseases affecting the vocal cords. However, many of these SSIs are still deemed as impractical due to a high degree of intrusiveness and discomfort, hence limiting their transition to outside of the laboratory environment. We aim to address the hardware challenges faced in developing a practical SSI for post-laryngectomy speech rehabilitation. A new Permanent Magnet Articulography (PMA) system is presented which fits within the palatal cavity of the user's mouth, giving unobtrusive appearance and high portability. The prototype is comprised of a miniaturized circuit constructed using commercial off-the-shelf (COTS) components and is implemented in the form of a dental retainer, which is mounted under roof of the user's mouth and firmly clasps onto the upper teeth. Preliminary evaluation via speech recognition experiments demonstrates that the intraoral prototype achieves reasonable word recognition accuracy and is comparable to the external PMA version. Moreover, the intraoral design is expected to improve on its stability and robustness, with a much improved appearance since it can be completely hidden inside the user's mouth. [8]. To date, a number of SSIs have been proposed in an attempt to extract non-acoustic information generated during speech production and reproduce audible speech using different sensing modalities, such as measuring electrical activities of the brain [9][10][11] or the articulator muscles [12][13][14], or by capturing movements of the speech articulators themselves [3,5,8,[15][16][17][18][19]. A comprehensive summary on different SSIs technologies were presented in [8]. Because of their unique feature, SSIs can also be deployed in acoustically challenging environment or where privacy/confidentially is desirable, and not limited to its use as a communication aid for speech impaired individuals. KeywordsDespite the attractive attributes of SSIs, there are still challenges in the form of hardware (e.g. portability, lightweight, unobtrusiveness and wearability) and processing software (e.g. efficiency, robustness and intelligibility speech generation). Preliminary discussions on the influential factors affecting the SSIs' implementation were presented in [8], based upon criteria such as ability to operate in silence and noisy environments, usability by laryngectomees, issue of invasiveness market reediness and cost.In the present work we employ the Permanent Magnet Articulography (PMA), which is a type SSI that is based on sensing the changes in the magnetic field generated by a set of permanent magnet markers attached onto the vocal apparatus (i.e. lips and tongue) during speech articulation by using an array of magnetic sensors located around the mouth [1,3]. Although PMA shares some similarities with Electromagnetic Articulography (EMA) [5,17], it does not explicitly provide the ...

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

“…On the other hand, PMA [ 42 , 43 , 44 , 45 , 46 ] is a technique to capture articulator displacement by using a permanent magnet on the tongue and detecting the magnetic signal using a wearable sensor. It is useful for speech recognition tasks, and the reported word detection accuracy is around 90% [ 42 ].…”

Biosignal Sensors and Deep Learning-Based Speech Recognition: A Review

“…Anxiety and depression are considered psychogenic causes of voice disorder [ 1 ]. Although there are several approaches for the treatment of voice disorder, such as PMA using articulatory data captured from the lips and tongue [ 2 ] or a contactless silent speech recognition system using an impulse radio ultra-wideband (IR-UWB) radar [ 3 ], the communication options are limited for the patients whose larynx (voice box) is surgically removed following throat cancer or trauma [ 4 ]. To overcome this problem, the external device, called electrolarynx, has been developed as a form of artificial larynx communication where an electronic vibration is produced and resulted in a monotonic sound that will be formed to speech [ 4 ].…”