FMG- and RNN-Based Estimation of Motor Intention of Upper-Limb Motion in Human-Robot Collaboration

Abstract

Research on human-robot interactions has been driven by the increasing employment of robotic manipulators in manufacturing and production. Toward developing more effective human-robot collaboration during shared tasks, this paper proposes an interaction scheme by employing machine learning algorithms to interpret biosignals acquired from the human user and accordingly planning the robot reaction. More specifically, a force myography (FMG) band was wrapped around the user's forearm and was used to collect information about muscle contractions during a set of collaborative tasks between the user and an industrial robot. A recurrent neural network model was trained to estimate the user's hand movement pattern based on the collected FMG data to determine whether the performed motion was random or intended as part of the predefined collaborative tasks. Experimental evaluation during two practical collaboration scenarios demonstrated that the trained model could successfully estimate the category of hand motion, i.e., intended or random, such that the robot either assisted with performing the task or changed its course of action to avoid collision. Furthermore, proximity sensors were mounted on the robotic arm to investigate if monitoring the distance between the user and the robot had an effect on the outcome of the collaborative effort. While further investigation is required to rigorously establish the safety of the human worker, this study demonstrates the potential of FMG-based wearable technologies to enhance human-robot collaboration in industrial settings.

A Review of EMG-, FMG-, and EIT-Based Biosensors and Relevant Human–Machine Interactivities and Biomedical Applications

“…Different movements are encoded into different force images. By decoding these images, original motion information can be obtained, which has been widely used in gesture recognition [ 2 ], human–machine collaboration [ 16 ], prosthetic control [ 17 ], and operational force estimation [ 12 ].…”

“…To acquire effective biosignals, the sensor needs to be in close contact with the skin, and piezoresistive sensors have this characteristic. The most frequently used piezoresistive sensors are force-sensitive resistors (FSR), for instance, FSR 402 [ 27 , 28 , 29 ] and FSR 400 [ 16 , 30 ], which are based on resistive polymer thick film sensor (RPTF) technology. Because of their thin profile, flexibility, and low cost, they become a practical solution for prosthetic pressure measurement [ 31 ].…”

Prescient teleoperation of humanoid robots

“…For the above elements, they are a suitable and valid technique for predicting the operator's intention in our teleoperation problem. Other data-driven methods for predicting gestures as time-series could be used in principle, for example LSTM-RNN (29,30), at the expense of more training data and potentially less smooth movements.…”

Human–Robot Collaboration in 3D via Force Myography Based Interactive Force Estimations Using Cross-Domain Generalization

“…Therefore, FMG biosignals can be useful in detecting certain human activities. Recently, FMG signals were used to recognize intentional or random hand movement during a human robot collaboration (HRC) task with an industrial YUMI manipulator robot [4]. In this study, human intention was identified by implementing a recurrent neural network (RNN).…”