Control System of Collaborative Robotic Based on the Methods of Contactless Recognition of Human Actions

Abstract

Human-robot collaboration is a key concept in modern intelligent manufacturing. Traditional human-robot interfaces are quite difficult to control and require additional operator training. The development of an intuitive and native user interface is important for the unobstructed interaction of human and robot in production. The control system of collaborative robotics described in the work is focused on increasing productivity, ensuring safety and ergonomics, minimize the cognitive workload of the operator in the process of human-robot interaction using contactless recognition of human actions. The system uses elements of technical vision to get of input data from the user in the form of gesture commands. As a set of commands for control collaborative robotic complexes and training the method proposed in the work, we use the actions from the UTD-MHAD database. The gesture recognition method is based on deep learning technology. An artificial neural network extracts the skeleton joints of the human and describes their position relative to each other and the center of gravity of the whole skeleton. The received descriptors feed to the input of the classifier, where the assignment to a specific class occur. This approach allows reducing the error from the redundancy of the data feed at the input of the neural network.

Action recognition for the robotics and manufacturing automation using 3-D binary micro-block difference

“…This procedure of convolution frequency spectrum with three-dimensional space-time Gabor filters is presented in [17], illustrated in Figure 2 and proceeds as Algorithm 1. Description and formulas for extraction the frequency spectrum using a three-dimensional Fourier transform, generating and convolution bank of 3-D Gabor filters, the inverse threedimensional Fourier transform is described in more detail in works [17,20,22].…”

“…Fig. 4 The principle of dividing a video sequence into short video sequences of three frames Micro-block inside three-dimensional patches is built according to the principle of volumetric local binary pattern (VLBP) [22]. VLBP are an object type for describing the characteristics of an object in the space-time domain.…”

“…The human body skeleton is built on the basis of 16 key points, joints, namely: right ankle, right knee, right thigh, left thigh, left knee, left ankle, torso, neck, chin, crown of the head, right wrist, right elbow, right shoulder, left wrist, left elbow, left shoulder (Figure 9). The action recognition method based on the construction of a human skeleton is described in detail in [17,22,[32][33]. Fig.…”

“…Fig. 9 An example of building a skeleton for an image [17,22] from the Leeds Sports Pose Dataset test kit [34].…”

Action recognition algorithm from visual sensor data for contactless robot control systems

“…The main goal of the developed software is easy and understandable interaction with industrial robots due to the fact that most people don't have programming knowledge. Modern user interfaces using robot sensing technologies and neural networks make it possible to solve this problem [3][4].…”