Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities

Wang, You; Zhang, Ming; Wu, Ru Feng; Gao, Han; Yue, Meng; Luo, Zhiyuan; Li, Guang

doi:10.3390/brainsci10070442

“…More recently, hybrid EMG-DNN systems for EMG-based ASR were investigated in [233]. In [234], transfer learning was found to be beneficial for silent speech recognition from EMG signals by exploiting neural networks trained on a image classification task as powerful feature extraction models. More recently, in [235], an empirical study was conducted to investigate the effect of the number of sEMG channels in silent speech recognition.…”

Section: ) Ssis Based On Emg Signalsmentioning

confidence: 99%

Silent Speech Interfaces for Speech Restoration: A Review

Gonzalez-Lopez

¹

,

Gomez-Alanis

²

,

Donas

³

et al. 2020

View full text Add to dashboard Cite

This review summarises the status of silent speech interface (SSI) research. SSIs rely on non-acoustic biosignals generated by the human body during speech production to enable communication whenever normal verbal communication is not possible or not desirable. In this review, we focus on the first case and present latest SSI research aimed at providing new alternative and augmentative communication methods for persons with severe speech disorders. SSIs can employ a variety of biosignals to enable silent communication, such as electrophysiological recordings of neural activity, electromyographic (EMG) recordings of vocal tract movements or the direct tracking of articulator movements using imaging techniques. Depending on the disorder, some sensing techniques may be better suited than others to capture speech-related information. For instance, EMG and imaging techniques are well suited for laryngectomised patients, whose vocal tract remains almost intact but are unable to speak after the removal of the vocal folds, but fail for severely paralysed individuals. From the biosignals, SSIs decode the intended message, using automatic speech recognition or speech synthesis algorithms. Despite considerable advances in recent years, most present-day SSIs have only been validated in laboratory settings for healthy users. Thus, as discussed in this paper, a number of challenges remain to be addressed in future research before SSIs can be promoted to real-world applications. If these issues can be addressed successfully, future SSIs will improve the lives of persons with severe speech impairments by restoring their communication capabilities.

show abstract

“…More recently, hybrid EMG-DNN systems for EMG-based ASR were investigated in [233]. In [234], transfer learning was found to be beneficial for silent speech recognition from EMG signals by exploiting neural networks trained on a image classification task as powerful feature extraction models. More recently, in [235], an empirical study was conducted to investigate the effect of the number of sEMG channels in silent speech recognition.…”

Section: B Muscle Activitymentioning

confidence: 99%

Silent Speech Interfaces for Speech Restoration: A Review

Gonzalez-Lopez,

Gomez-Alanis,

Martín-Doñas

et al. 2020

Preprint

0

View full text Add to dashboard Cite

This review summarises the status of silent speech interface (SSI) research. SSIs rely on non-acoustic biosignals generated by the human body during speech production to enable communication whenever normal verbal communication is not possible or not desirable. In this review, we focus on the first case and present latest SSI research aimed at providing new alternative and augmentative communication methods for persons with severe speech disorders. SSIs can employ a variety of biosignals to enable silent communication, such as electrophysiological recordings of neural activity, electromyographic (EMG) recordings of vocal tract movements or the direct tracking of articulator movements using imaging techniques. Depending on the disorder, some sensing techniques may be better suited than others to capture speech-related information. For instance, EMG and imaging techniques are well suited for laryngectomised patients, whose vocal tract remains almost intact but are unable to speak after the removal of the vocal folds, but fail for severely paralysed individuals. From the biosignals, SSIs decode the intended message, using automatic speech recognition or speech synthesis algorithms. Despite considerable advances in recent years, most present-day SSIs have only been validated in laboratory settings for healthy users. Thus, as discussed in this paper, a number of challenges remain to be addressed in future research before SSIs can be promoted to real-world applications. If these issues can be addressed successfully, future SSIs will improve the lives of persons with severe speech impairments by restoring their communication capabilities.

show abstract

“…Commonly used methods are wavelet analysis [8,9], fourier spectral analysis [10], empirical mode decomposition (EMD) [11,12] and other feature transformation techniques [13][14][15]. However, exquisite technology and rich expert experience are required in the above approaches [16]. Pattern recognition is to identify the fault information within the extracted features by artificial intelligence method and realize automatic fault diagnosis.…”

Section: Introductionmentioning

confidence: 99%

A Novel Anti-Noise Fault Diagnosis Approach for Rolling Bearings Based on Convolutional Neural Network Fusing Frequency Domain Feature Matching Algorithm

Zhou

¹

,

Mao

²

,

Li

³

2021

Sensors

View full text Add to dashboard Cite

The development of deep learning provides a new research method for fault diagnosis. However, in the industrial field, the labeled samples are insufficient and the noise interference is strong so that raw data obtained by the sensor are occupied with noise signal. It is difficult to recognize time-domain fault signals under the severe noise environment. In order to solve these problems, the convolutional neural network (CNN) fusing frequency domain feature matching algorithm (FDFM), called CNN-FDFM, is proposed in this paper. FDFM extracts key frequency features from signals in the frequency domain, which can maintain high accuracy in the case of strong noise and limited samples. CNN automatically extracts features from time-domain signals, and by using dropout to simulate noise input and increasing the size of the first-layer convolutional kernel, the anti-noise ability of the network is improved. Softmax with temperature parameter T and D-S evidence theory are used to fuse the two models. As FDFM and CNN can provide different diagnostic information in frequency domain, and time domain, respectively, the fused model CNN-FDFM achieves higher accuracy under severe noise environment. In the experiment, when a signal-to-noise ratio (SNR) drops to -10 dB, the diagnosis accuracy of CNN-FDFM still reaches 93.33%, higher than CNN’s accuracy of 45.43%. Besides, when SNR is greater than -6 dB, the accuracy of CNN-FDFM is higher than 99%.

show abstract

Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities

Cited by 27 publications

References 43 publications

Silent Speech Interfaces for Speech Restoration: A Review

Silent Speech Interfaces for Speech Restoration: A Review

Silent Speech Interfaces for Speech Restoration: A Review

A Novel Anti-Noise Fault Diagnosis Approach for Rolling Bearings Based on Convolutional Neural Network Fusing Frequency Domain Feature Matching Algorithm

Contact Info

Product

Resources

About