Convolutional neural network (CNN) has been widely exploited for simultaneous and proportional myoelectric control due to its capability of deriving informative, representative and transferable features from surface electromyography (sEMG). However, muscle contractions have strong temporal dependencies but conventional CNN can only exploit spatial correlations. Considering that long short-term memory neural network (LSTM) is able to capture long-term and non-linear dynamics of time-series data, in this paper we propose a CNN-LSTM hybrid model to fully explore the temporal-spatial information in sEMG. Firstly, CNN is utilized to extract deep features from sEMG spectrum, then these features are processed via LSTM-based sequence regression to estimate wrist kinematics. Six healthy participants are recruited for the participatory collection and motion analysis under various experimental setups. Estimation results in both intra-session and inter-session evaluations illustrate that CNN-LSTM significantly outperforms CNN, LSTM and several representative machine learning approaches, particularly when complex wrist movements are activated. Index Terms-sEMG, wrist kinematics estimation, deep learning, convolutional neural network, long short-term memory network, hybrid model.
In the past decades, classical machine learning (ML) methods have been widely investigated in wrist kinematics estimation for the control of prosthetic hands. Currently deeper structures have shown great potential to further improve prediction accuracy. In this paper we present a single stream convolutional neural network (CNN) for mapping surface electromyography (sEMG) to wrist angles within three degrees-of-freedom (DOFs). Two types of two dimensional (2D) sEMG images are constructed in time domain and spectrum as CNN inputs, respectively. Six typical linear and nonlinear ML models are implemented for comparison, where four efficient time-spatial hand-crafted features are extracted to represent feature engineering. Experiment results with four able-bodied participants illustrate that CNN with 2D spectrum sEMG images can achieve highest accuracy in most testing sessions. In other sessions, it is still competitive to the most promising ML techniques. The core strength of deep learning (DL), i.e. feature learning via deep structures and efficient algorithms, is verified to be more powerful than classical feature engineering, particularly in smaller datasets.
Recently, convolutional neural network (CNN) has been widely investigated to decode human intentions using surface Electromyography (sEMG) signals. However, a pre-trained CNN model usually suffers from severe degradation when testing on a new individual, and this is mainly due to domain shift where characteristics of training and testing sEMG data differ substantially. To enhance inter-subject performances of CNN in the wrist kinematics estimation, we propose a novel regression scheme for supervised domain adaptation (SDA), based on which domain shift effects can be effectively reduced. Specifically, a two-stream CNN with shared weights is established to exploit source and target sEMG data simultaneously, such that domaininvariant features can be extracted. To tune CNN weights, both regression losses and a domain discrepancy loss are employed, where the former enable supervised learning and the latter minimizes distribution divergences between two domains. In this study, eight healthy subjects were recruited to perform wrist flexion-extension movements. Experiment results illustrated that the proposed regression SDA outperformed fine-tuning, a stateof-the-art transfer learning method, in both single-single and multiple-single scenarios of kinematics estimation. Unlike finetuning which suffers from catastrophic forgetting, regression SDA can maintain much better performances in original domains, which boosts the model reusability among multiple subjects.
To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.
Grasp classification using data gloves can enable therapists to monitor patients efficiently by providing concise information about the activities performed by these patients. Although, classical machine learning algorithms have been applied in grasp classification, they require manual feature extraction to achieve high accuracy. In contrast, convolutional neural networks (CNNs) have outperformed popular machine learning algorithms in several classification scenarios because of their ability to extract features automatically from raw data. However, they have not been implemented on grasp classification using a data glove. In this study, we apply a CNN in grasp classification using a piezoresistive textile data glove knitted from conductive yarn and an elastomeric yarn. The data glove was used to collect data from five participants who grasped thirty objects each following Schlesinger's taxonomy. We investigate a CNN's performance in two scenarios where the validation objects are known and unknown. Our results show that a simple CNN architecture outperformed k-nn, Gaussian SVM, and Decision Tree algorithms in both scenarios in terms of the classification accuracy.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.