Using Inferred Gestures from sEMG Signal to Teleoperate a Domestic Robot for the Disabled

Nasri, Nadia; Gomez-Donoso, Francisco; Orts-Escolano, Sergio; Cazorla, Miguel

doi:10.1007/978-3-030-20518-8_17

Cited by 5 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As mentioned in the introduction, most deep learning methods are based on the sEMG signals. The mainstream methods can be classified into three categories: CNN models [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ], RNN models [ 18 , 19 , 20 , 21 , 22 , 23 , 24 ], and ANN models [ 25 , 26 , 27 , 28 ]. The single-layer CNN proposed by Zia ur Rehman M accomplished the classification task of 7 gestures [ 12 ], which pioneered the application of CNN models to gesture recognition.…”

Section: Related Workmentioning

confidence: 99%

“…Many studies have proposed machine learning or deep learning algorithms to implement Myo-based gesture recognition tasks. Among them, support vector machine (SVM) [ 1 , 2 , 3 , 4 ], k-nearest neighbor (KNN) [ 5 , 6 , 7 , 8 , 9 ], decision tree (DT) [ 10 ], convolutional neural network (CNN) [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ], recurrent neural network (RNN) [ 18 , 19 , 20 , 21 , 22 , 23 , 24 ], and artificial neural networks (ANN) [ 25 , 26 , 27 , 28 , 29 , 30 ] are the most popular algorithms with good recognition accuracy; however, there are still some challenges in this field of research. First, most studies build their datasets for specific application scenarios; these datasets involve mostly less than 10 gesture actions, and there is a lack of publicly available datasets for more classification tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Jiang

Song

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models’ test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Jiang

Song

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…However in the present work, we decided to use a sub-dataset of the dataset acquired in our previous work, which was recorded from hand gestures using low-cost sensors. Our dataset consisted of static hand gestures and we decided to study the behavior of a system with a smaller training group than in previous works [ 40 , 41 ].…”

Section: Related Workmentioning

confidence: 99%

“…Hence, in the present work, we decided to expand the number of gestures through the use of deep learning techniques. In our previous work [ 40 ], we examined the efficiency of gated recurrent unit (GRU) architecture [ 49 ] on raw sEMG signals for six gestures and created an application for controlling domestic robots to help disabled people [ 41 ].…”

Section: Related Workmentioning

confidence: 99%

An sEMG-Controlled 3D Game for Rehabilitation Therapies: Real-Time Time Hand Gesture Recognition Using Deep Learning Techniques

Nasri

Orts-Escolano

Cazorla

2020

Sensors

Self Cite

View full text Add to dashboard Cite

In recent years the advances in Artificial Intelligence (AI) have been seen to play an important role in human well-being, in particular enabling novel forms of human-computer interaction for people with a disability. In this paper, we propose a sEMG-controlled 3D game that leverages a deep learning-based architecture for real-time gesture recognition. The 3D game experience developed in the study is focused on rehabilitation exercises, allowing individuals with certain disabilities to use low-cost sEMG sensors to control the game experience. For this purpose, we acquired a novel dataset of seven gestures using the Myo armband device, which we utilized to train the proposed deep learning model. The signals captured were used as an input of a Conv-GRU architecture to classify the gestures. Further, we ran a live system with the participation of different individuals and analyzed the neural network’s classification for hand gestures. Finally, we also evaluated our system, testing it for 20 rounds with new participants and analyzed its results in a user study.

show abstract

“…In 1988, the first documented robotic-assisted surgical procedure was performed with a robotic arm and a computerized tomography (CT) scanner for a CT-guided brain tumor biopsy [6]. Since then, technological advancements have greatly increased trust in robot capabilities and they have helped people with both their mental and physical needs [7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Assistive Robot with an AI-Based Application for the Reinforcement of Activities of Daily Living: Technical Validation with Users Affected by Neurodevelopmental Disorders

et al. 2022

View full text Add to dashboard Cite

In this work, we propose the first study of a technical validation of an assistive robotic platform, which has been designed to assist people with neurodevelopmental disorders. The platform is called LOLA2 and it is equipped with an artificial intelligence-based application to reinforce the learning of daily life activities in people with neurodevelopmental problems. LOLA2 has been integrated with an ROS-based navigation system and a user interface for healthcare professionals and their patients to interact with it. Technically, we have been able to embed all these modules into an NVIDIA Jetson Xavier board, as well as an artificial intelligence agent for online action detection (OAD). This OAD approach provides a detailed report on the degree of performance of a set of daily life activities that are being learned or reinforced by users. All the human–robot interaction process to work with users with neurodevelopmental disorders has been designed by a multidisciplinary team. Among its main features are the ability to control the robot with a joystick, a graphical user interface application that shows video tutorials with the activities to reinforce or learn, and the ability to monitor the progress of the users as they complete tasks. The main objective of the assistive robotic platform LOLA2 is to provide a system that allows therapists to track how well the users understand and perform daily tasks. This paper focuses on the technical validation of the proposed platform and its application. To do so, we have carried out a set of tests with four users with neurodevelopmental problems and special physical conditions under the supervision of the corresponding therapeutic personnel. We present detailed results of all interventions with end users, analyzing the usability, effectiveness, and limitations of the proposed technology. During its initial technical validation with real users, LOLA2 was able to detect the actions of users with disabilities with high precision. It was able to distinguish four assigned daily actions with high accuracy, but some actions were more challenging due to the physical limitations of the users. Generally, the presence of the robot in the therapy sessions received excellent feedback from medical professionals as well as patients. Overall, this study demonstrates that our developed robot is capable of assisting and monitoring people with neurodevelopmental disorders in performing their daily living tasks.

show abstract

Using Inferred Gestures from sEMG Signal to Teleoperate a Domestic Robot for the Disabled

Cited by 5 publications

References 25 publications

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

An sEMG-Controlled 3D Game for Rehabilitation Therapies: Real-Time Time Hand Gesture Recognition Using Deep Learning Techniques

Assistive Robot with an AI-Based Application for the Reinforcement of Activities of Daily Living: Technical Validation with Users Affected by Neurodevelopmental Disorders

Contact Info

Product

Resources

About