Gesture recognition based teleoperation framework of robotic fish

Mi, Jinpeng; Sun, Yu; Wang, Yu; Deng, Zhen; Li, Liang; Zhang, Jianwei; Xie, Guanming

doi:10.1109/robio.2016.7866311

Cited by 10 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2. We then take the cross product of the Affine Matrix from Forward Kinematics in equation (2) and Translation Matrix of the CoM of individual links from Table II in equation ( 1). 3.…”

Section: Whole-body Commentioning

confidence: 99%

See 1 more Smart Citation

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

et al. 2022

View full text Add to dashboard Cite

The capabilities of teleoperated robots can be enhanced with the ability to recognise and reproduce human-like behaviour. The proposed framework presents motion recognition for a Kinect-based NAO teleoperation. It allows the NAO robot to recognise the human motions and act as a human motion imitator. A stable whole-body imitation is still a challenging issue because of the difficulty in dynamic balancing of centre of mass (CoM). In this paper, a novel adaptive balancing technique for NAO (ABTN) is proposed to control the whole body in single as well as double supporting phases. It targets dynamic balancing of the humanoid robot by solving forward kinematics and applying a weighted average of mass with the CoMs of individual links with respect to the previous joint frames, which provides us with the dynamic CoM of the whole body. Our novel approach uses this dynamic CoM and calculates joint angles using proposed pitch and roll control algorithm to keep the dynamic CoM inside the stable region. Additionally, the NAO robot is capable of recognising human motions using the proposed 7-layer one-dimensional convolutional neural network (1D-CNN). To solve the problem of variable length of time sequences, Zero padding is introduced with 1D-CNN. It attains a recognition accuracy of 95% as compared to the hidden Markov model and neural network. The experimental results demonstrate that the developed teleoperation framework is robust and serves as potential support for the development and application of teleoperated robots.

show abstract

“…2. We then take the cross product of the Affine Matrix from Forward Kinematics in equation (2) and Translation Matrix of the CoM of individual links from Table II in equation ( 1). 3.…”

Section: Whole-body Commentioning

confidence: 99%

“…The teleoperation bridges the gap between humans and the robots. The controlling of robotic operations from a distance is termed as teleoperation [2]. The distance varies from billions of kilometres as in space applications to centimetres as in micro-applications or microsurgery.…”

Section: Introductionmentioning

confidence: 99%

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…This section reviews the recent studies conducted in the field of gesture recognition. Moreover, active developments in gesture recognition techniques have led to the recent developments in virtual reality (VR) technology [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]. The gesture recognition accuracy, which is a key metric to evaluate the effectiveness of gesture recognition systems, has witnessed steady improvements using input data from multiple sensors [17][18][19][20][21][22].…”

Section: Related Workmentioning

confidence: 99%

“…Table 2 represents available joints acquired by each sensor. 11,4 ), Thumb proximal (s 11,5 ), Thumb metacarpal (s 11,6 ) Index distal (s 11,7 ), Index intermediate (s 11,8 ), Index proximal (s 11,9 ), Index metacarpal (s 11,10 ) Middle distal (s 11,11 ), Middle intermediate (s 11,12 ), Middle proximal (s 11,13 ), Middle metacarpal (s 11,14 ) Ring distal (s 11,15 ), Ring intermediate (s 11,16 ), Ring proximal (s 11,17 ), Ring metacarpal (s 11,18 ) Pinky distal (s 11,19 ), Pinky intermediate (s 11,20 ), Pinky proximal (s 11,21 When the gesture recording module records and stores a gesture, the gesture at time t, g t , includes B * t and S t at time t, g t = B * t , S t . Although the proposed method stores S t , S * t ⊂ S t ⊂S, during the gesture registration stage, the learning model learns and recognizes only S * t during the gesture learning and recognition stage in order to reduce the processing time required.…”

Section: Gesture Registration Stagementioning

confidence: 99%

Advanced Machine Learning for Gesture Learning and Recognition Based on Intelligent Big Data of Heterogeneous Sensors

et al. 2019

View full text Add to dashboard Cite

With intelligent big data, a variety of gesture-based recognition systems have been developed to enable intuitive interaction by utilizing machine learning algorithms. Realizing a high gesture recognition accuracy is crucial, and current systems learn extensive gestures in advance to augment their recognition accuracies. However, the process of accurately recognizing gestures relies on identifying and editing numerous gestures collected from the actual end users of the system. This final end-user learning component remains troublesome for most existing gesture recognition systems. This paper proposes a method that facilitates end-user gesture learning and recognition by improving the editing process applied on intelligent big data, which is collected through end-user gestures. The proposed method realizes the recognition of more complex and precise gestures by merging gestures collected from multiple sensors and processing them as a single gesture. To evaluate the proposed method, it was used in a shadow puppet performance that could interact with on-screen animations. An average gesture recognition rate of 90% was achieved in the experimental evaluation, demonstrating the efficacy and intuitiveness of the proposed method for editing visualized learning gestures.

show abstract

“…Gestures support and make easy communication with others by using body parts such as face and hand. The use of gestures varies in different areas such as in-vehicle interaction (Molchanov et al, 2015), virtual-reality (Sagayam andHemanth, 2017), control of prosthesis (Guo et al, 2017), augmented reality (Liang et al, 2015), teleoperation (Mi et al, 2016) and sign language (Zhang et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Machine Learning Based Hand Gesture Recognition via EMG Data

Şentürk

Bakay

2021

ADCAIJ

View full text Add to dashboard Cite

Electromyography (EMG) data gives information about the electrical activity related to muscles. EMG data obtained from arm through sensors helps to understand hand gestures. For this work, hand gesture data were taken from UCI2019 EMG dataset obtained from MYO thalmic armband were classied with six dierent machine learning algorithms. Articial Neural Network (ANN), Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Naive Bayes (NB), Decision Tree (DT) and Random Forest (RF) methods were preferred for comparison based on several performance metrics which are accuracy, precision, sensitivity, specicity, classication error, kappa, root mean squared error (RMSE) and correlation. The data belongs to seven hand gestures. 700 samples from 7 classes (100 samples per group) were used in the experiments. The splitting ratio in the classication was 0.8-0.2, i.e. 80% of the samples were used in training and 20% of data were used in testing phase of the classier. NB was found to be the best among other methods because of high accuracy (96.43%) and sensitivity (96.43%) and the lowest RMSE (0.189). Considering the results of the performance parameters, it can be said that this study recognizes and classies seven hand gestures successfully in comparison with the literature.

show abstract

Gesture recognition based teleoperation framework of robotic fish

Cited by 10 publications

References 15 publications

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

Advanced Machine Learning for Gesture Learning and Recognition Based on Intelligent Big Data of Heterogeneous Sensors

Machine Learning Based Hand Gesture Recognition via EMG Data

Contact Info

Product

Resources

About