Deep Learning for EMG-based Human-Machine Interaction: A Review

Xiong, Dezhen; Zhang, Daohui; Zhao, Xingang; Zhao, Yiwen

doi:10.1109/jas.2021.1003865

Cited by 217 publications

(110 citation statements)

References 161 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, sEMG signals have a user-specific nature, causing the amplitude and frequencies to be highly variable among individuals even when signals are measured from the same location with the same motion [18]. Although it has been reported that features learned by deep neural networks may be able to share similar distributions across different subjects [11,19], the inter-subject problem can still lead to a sharp decline in the estimation performance of the previously trained model [11,20].…”

Section: Introductionmentioning

confidence: 99%

Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG

Bao

Zaidi

Xie

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Recently, convolutional neural network (CNN) has been widely investigated to decode human intentions using surface Electromyography (sEMG) signals. However, a pre-trained CNN model usually suffers from severe degradation when testing on a new individual, and this is mainly due to domain shift where characteristics of training and testing sEMG data differ substantially. To enhance inter-subject performances of CNN in the wrist kinematics estimation, we propose a novel regression scheme for supervised domain adaptation (SDA), based on which domain shift effects can be effectively reduced. Specifically, a two-stream CNN with shared weights is established to exploit source and target sEMG data simultaneously, such that domaininvariant features can be extracted. To tune CNN weights, both regression losses and a domain discrepancy loss are employed, where the former enable supervised learning and the latter minimizes distribution divergences between two domains. In this study, eight healthy subjects were recruited to perform wrist flexion-extension movements. Experiment results illustrated that the proposed regression SDA outperformed fine-tuning, a stateof-the-art transfer learning method, in both single-single and multiple-single scenarios of kinematics estimation. Unlike finetuning which suffers from catastrophic forgetting, regression SDA can maintain much better performances in original domains, which boosts the model reusability among multiple subjects.

show abstract

Section: Introductionmentioning

confidence: 99%

Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG

Bao

Zaidi

Xie

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…Recently, many mathematical methods for analyzing EEG, EMG, and tremor signals have been developed. Historically, EMG analysis methods evolved from spectral analysis [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] and time-domain signal analysis methods such as morphological analysis [ 8 ], amplitude analysis [ 9 ], and autoregressive analysis [ 10 , 11 ] towards time–frequency domain analysis [ 12 , 13 , 14 , 15 , 16 ]. The state-of-the-art of EMG analysis methods is characterized by the active use of nonlinear data analysis methods [ 17 ], such as fractal analysis [ 18 ], phase analysis [ 19 ], recurrent quantification analysis [ 4 , 20 , 21 ], and the deep learning of neural networks [ 12 , 22 , 23 , 24 , 25 ].…”

Section: Introductionmentioning

confidence: 99%

“…Historically, EMG analysis methods evolved from spectral analysis [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] and time-domain signal analysis methods such as morphological analysis [ 8 ], amplitude analysis [ 9 ], and autoregressive analysis [ 10 , 11 ] towards time–frequency domain analysis [ 12 , 13 , 14 , 15 , 16 ]. The state-of-the-art of EMG analysis methods is characterized by the active use of nonlinear data analysis methods [ 17 ], such as fractal analysis [ 18 ], phase analysis [ 19 ], recurrent quantification analysis [ 4 , 20 , 21 ], and the deep learning of neural networks [ 12 , 22 , 23 , 24 , 25 ]. According to the authors, the existing methods for analyzing EEG, EMG, and tremor signals, such as wavelet analysis [ 26 , 27 , 28 ], focus on local time–frequency changes in the signal and, therefore, do not reveal the generalized properties of the signal.…”

Section: Introductionmentioning

confidence: 99%

A Statistical Method for Exploratory Data Analysis Based on 2D and 3D Area under Curve Diagrams: Parkinson’s Disease Investigation

Sushkova

Morozov

Gabova

et al. 2021

Sensors

View full text Add to dashboard Cite

A statistical method for exploratory data analysis based on 2D and 3D area under curve (AUC) diagrams was developed. The method was designed to analyze electroencephalogram (EEG), electromyogram (EMG), and tremorogram data collected from patients with Parkinson’s disease. The idea of the method of wave train electrical activity analysis is that we consider the biomedical signal as a combination of the wave trains. The wave train is the increase in the power spectral density of the signal localized in time, frequency, and space. We detect the wave trains as the local maxima in the wavelet spectrograms. We do not consider wave trains as a special kind of signal. The wave train analysis method is different from standard signal analysis methods such as Fourier analysis and wavelet analysis in the following way. Existing methods for analyzing EEG, EMG, and tremor signals, such as wavelet analysis, focus on local time–frequency changes in the signal and therefore do not reveal the generalized properties of the signal. Other methods such as standard Fourier analysis ignore the local time–frequency changes in the characteristics of the signal and, consequently, lose a large amount of information that existed in the signal. The method of wave train electrical activity analysis resolves the contradiction between these two approaches because it addresses the generalized characteristics of the biomedical signal based on local time–frequency changes in the signal. We investigate the following wave train parameters: wave train central frequency, wave train maximal power spectral density, wave train duration in periods, and wave train bandwidth. We have developed special graphical diagrams, named AUC diagrams, to determine what wave trains are characteristic of neurodegenerative diseases. In this paper, we consider the following types of AUC diagrams: 2D and 3D diagrams. The technique of working with AUC diagrams is illustrated by examples of analysis of EMG in patients with Parkinson’s disease and healthy volunteers. It is demonstrated that new regularities useful for the high-accuracy diagnosis of Parkinson’s disease can be revealed using the method of analyzing the wave train electrical activity and AUC diagrams.

show abstract

“…In order to restore the patient's lost limb function or assist them for daily activities such as eating and drinking, artificial hands, exoskeletons, robotic arms, smart wheelchairs and other assistive robots have emerged (Wu et al, 2018;Kaur, 2021). How to establish a natural, efficient and stable human-computer interface (HCI) has become a difficult and hot point in the research of interactive control of rehabilitation aids (Mussa-Ivaldi et al, 2013;Venkatakrishnan et al, 2014;Gordleeva et al, 2020;Xiong et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

“…Compared with lower recognition accuracy or need additional stimulation for EEG-based HCI (such as motor imagery and steady state visual evoked potential) (Lin et al, 2016;Chu et al, 2018) and relative fewer recognizable intentions for EOG-based HCI (Bastos-Filho et al, 2014;He and Li, 2017) or hybrid gaze-brain machine interface (Li et al, 2017;Krausz et al, 2020;Zeng et al, 2020), EMG-based HCI has been widely used in the field of neurorehabilitation with the advantage of higher accuracy and stability, especially for decoding motor intentions of the limb with EMG (Ding et al, 2015;Hussain et al, 2016;Zhang et al, 2019). However, intent recognition based on limb EMG is still facing a huge challenge due to the abnormal signal in the absence of limb function (Jaramillo-Yánez et al, 2020;Xiong et al, 2021). Hence, instead of limb EMG, a novel intention recognition method based on facial electromyography (fEMG) and the HCI based on fEMG have been paid attention and partly researched (Hamedi et al, 2011;Tamura et al, 2012;Bastos-Filho et al, 2014;Nam et al, 2014;Inzelberg et al, 2018;Kapur et al, 2018Kapur et al, , 2020.…”

Section: Introductionmentioning

confidence: 99%

Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback

et al. 2021

Self Cite

View full text Add to dashboard Cite

Patients who have lost limb control ability, such as upper limb amputation and high paraplegia, are usually unable to take care of themselves. Establishing a natural, stable, and comfortable human-computer interface (HCI) for controlling rehabilitation assistance robots and other controllable equipments will solve a lot of their troubles. In this study, a complete limbs-free face-computer interface (FCI) framework based on facial electromyography (fEMG) including offline analysis and online control of mechanical equipments was proposed. Six facial movements related to eyebrows, eyes, and mouth were used in this FCI. In the offline stage, 12 models, eight types of features, and three different feature combination methods for model inputing were studied and compared in detail. In the online stage, four well-designed sessions were introduced to control a robotic arm to complete drinking water task in three ways (by touch screen, by fEMG with and without audio feedback) for verification and performance comparison of proposed FCI framework. Three features and one model with an average offline recognition accuracy of 95.3%, a maximum of 98.8%, and a minimum of 91.4% were selected for use in online scenarios. In contrast, the way with audio feedback performed better than that without audio feedback. All subjects completed the drinking task in a few minutes with FCI. The average and smallest time difference between touch screen and fEMG under audio feedback were only 1.24 and 0.37 min, respectively.

show abstract

Deep Learning for EMG-based Human-Machine Interaction: A Review

Cited by 217 publications

References 161 publications

Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG

Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG

A Statistical Method for Exploratory Data Analysis Based on 2D and 3D Area under Curve Diagrams: Parkinson’s Disease Investigation

Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback

Contact Info

Product

Resources

About