Development of a Verbal Robot Hand Gesture Recognition System

Abstract

This article analyzes the most famous sign languages, the correlation of sign languages, and also considers the development of a verbal robot hand gesture recognition system in relation to the Kazakh language. The proposed system contains a touch sensor, in which the contact of the electrical property of the user's skin is measured, which provides more accurate information for simulating and indicating the gestures of the robot hand. Within the framework of the system, the speed and accuracy of recognition of each gesture of the verbal robot are calculated. The average recognition accuracy was over 98%. The detection time was 3ms on a 1.9 GHz Jetson Nano processor, which is enough to create a robot showing natural language gestures. A complete fingerprint of the Kazakh sign language for a verbal robot is also proposed. To improve the quality of gesture recognition, a machine learning method was used. The operability of the developed technique for recognizing gestures by a verbal robot was tested, and on the basis of computational experiments, the effectiveness of algorithms and software for responding to a verbal robot to a voice command was evaluated based on automatic recognition of a multilingual human voice. Thus, we can assume that the authors have proposed an intelligent verbal complex implemented in Python with the CMUSphinx communication module and the PyOpenGL graphical command execution simulator. Robot manipulation module based on 3D modeling from ABB.

Developing application techniques of kinematics and simulation model for InMoov robot

“…In [14], the authors investigated the sagittal and frontal planes to obtain closed form solutions for the inverse and forward kinematics of the humanoid robot. In [15][16][17], the authors, using three variables, manipulated both sides of the kinematic matrix equation to express the motion of one leg.…”

“…The proposed work was executed on Python Keras platform. Herein, learning data sets were generated, using equations ( 12)- (17). First, there was generated the set of 40000 data in compliance with the formula for input parameters of P x , P y and P z coordinates in millimeters.…”