Towards optimizing electrode configurations for silent speech recognition based on high-density surface electromyography

Abstract

Objective. Silent speech recognition (SSR) based on surface electromyography (sEMG) is an attractive non-acoustic modality of human-machine interfaces that convert the neuromuscular electrophysiological signals into computer-readable textual messages. The speaking process involves complex neuromuscular activities spanning a large area over the facial and neck muscles, thus the locations of the sEMG electrodes considerably affected the performance of the SSR system. However, most of the previous studies used only a quite limited number of electrodes that were placed empirically without prior quantitative analysis, resulting in uncertainty and unreliability of the SSR outcomes. Approach. In this study, the technique of high-density sEMG was proposed to provide a full representation of the articulatory muscle activities so that the optimal electrode configuration for SSR could be systemically explored. A total of 120 closely spaced electrodes were placed on the facial and neck muscles to collect the high-density sEMG signals for classifying ten digits (0–9) silently spoken in both English and Chinese. The sequential forward selection algorithm was adopted to explore the optimal electrodes configurations. Main Results. The results showed that the classification accuracy increased rapidly and became saturated quickly when the number of selected electrodes increased from 1 to 120. Using only ten optimal electrodes could achieve a classification accuracy of 86% for English and 94% for Chinese, whereas as many as 40 non-optimized electrodes were required to obtain comparable accuracies. Also, the optimally selected electrodes seemed to be mostly distributed on the neck instead of the facial region, and more electrodes were required for English recognition to achieve the same accuracy. Significance. The findings of this study can provide useful guidelines about electrode placement for developing a clinically feasible SSR system and implementing a promising approach of human-machine interface, especially for patients with speaking difficulties.

Customizable, reconfigurable, and anatomically coordinated large-area, high-density electromyography from drawn-on-skin electrode arrays

“…1C ). The arrays were even matched to the complex arrangement and shapes of some facial muscles, including the zygomaticus and risorius muscles ( 8 , 41 ). Although the DoS MEAs here are shown on a healthy subject, they could easily be adapted to the limb of an amputee patient, unlike the conventional planar and flexible grids ( 10 ).…”

“…Customizing the electrodes to the muscle anatomy can offer the appropriate resolution ( 1 , 41 , 47 ) and better classification accuracy from pattern recognition algorithms without creating redundancies ( 3 , 8 ). Redundancies in EMG data are interference signals that decrease the differentiability of the data for classification.…”

“…The typically utilized conventional high-density MEAs are indiscriminate to the spatial arrangement of muscles with varying geometries and cannot be reconfigured in situ to the appropriate number and specific positions of electrodes to offer the most informative data, which is a significant challenge to overcome ( 14 , 16 , 17 ). In addition to highly redundant data/missed information ( 8 , 18–20 ), the anatomical mismatch between the existing MEAs and target muscles also results in electrode shifts ( 21 , 22 ) and motion artifacts ( 23 , 24 ), further reducing the overall quality of surface EMG mapping. Devices with more deformable electrodes ( 25–35 ) could potentially be useful or repurposed for reconfiguration to some extent, but they were not designed nor are readily feasible to particularly solve the anatomical mismatch issue.…”

Laser-Induced Graphene Strain Sensor for Conformable Lip-Reading Recognition and Human–Machine Interaction

“…However, it is unsuitable for the speech-impaired and is susceptible to environmental noise interference. In contrast, lip-reading recognition (LRR) technology, which enables silent speech recognition, has gained significant research interest in various fields, including speech-impaired communication and human–machine interaction (HMI). , …”

Decoding Silent Speech Based on High-Density Surface Electromyogram Using Spatiotemporal Neural Network

“…In addition, with the development of biosensing technology, high-density (HD) electrode array is widely applied to simultaneously record multichannel sEMG signals from a number of target muscles or muscle groups in relatively large areas, and the use of HD-sEMG signals has been proved to promote the development of myoelectric control system as well [23]- [25]. Some researchers applied the HD electrode arrays to the sEMG-based SSR, demonstrating that their spatial information of muscle activities can improve the SSR performance [26]- [28]. It was worth note that the HD-sEMG data recorded by two-dimensional electrode arrays can be viewed as an image, where discriminative spatial features can be well characterized by deep image processing techniques such as convolutional neural networks (CNNs) [29].…”