A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (<0.1 mm2) could effectively capture the biaxial strain information with high reliability. We attached four strain sensors near the subject’s mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system’s reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited a relatively low accuracy rate (42.60%).
Background: Temporomandibular joint disorder (TMD), which is a broad category encompassing disc displacement, is a common condition with an increasing prevalence. This study aimed to develop an automated movement tracing algorithm for mouth opening and closing videos, and to quantitatively analyze the relationship between the results obtained using this developed system and disc position on magnetic resonance imaging (MRI).Methods: Mouth opening and closing videos were obtained with a digital camera from 91 subjects, who underwent MRI. Before video acquisition, an 8.0-mm-diameter circular sticker was attached to the center of the subject's upper and lower lips. The automated mouth opening tracing system based on computer vision was developed in two parts: (I) automated landmark detection of the upper and lower lips in acquired videos, and (II) graphical presentation of the tracing results for detected landmarks and an automatically calculated graph height (mouth opening length) and width (sideways values). The graph paths were divided into three types: straight, sideways-skewed, and limited-straight line graphs. All traced results were evaluated according to disc position groups determined using MRI. Graph height and width were compared between groups using analysis of variance (SPSS version 25.0; IBM Corp., Armonk, NY, USA).Results: Subjects with a normal disc position predominantly (85.72%) showed straight line graphs.The other two types (sideways-skewed or limited-straight line graphs) were found in 85.0% and 89.47% in the anterior disc displacement with reduction group and in the anterior disc displacement without reduction group, respectively, reflecting a statistically significant correlation (χ 2 =38.113, P<0.001). A statistically significant difference in graph height was found between the normal group and the anterior disc displacement without reduction group, 44.90±9.61 mm and 35.78±10.24 mm, respectively (P<0.05).
Conclusions:The developed mouth opening tracing system was reliable.
A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, in this study, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (< 0.1 mm2) could capture the biaxial strain information with high reliability. We attached four strain sensors near the subject’s mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system’s reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited an accuracy rate of 35.00%.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.