A real-time human-robot interaction framework with robust background invariant hand gesture detection

Abstract

In the light of factories of the future, to ensure productive and safe interaction between robot and human coworkers, it is imperative that the robot extracts the essential information of the coworker. We address this by designing a reliable framework for real-time safe human-robot collaboration, using static hand gestures and 3D skeleton extraction. OpenPose library is integrated with Microsoft Kinect V2, to obtain a 3D estimation of the human skeleton. With the help of 10 volunteers, we recorded an image dataset of alphanumeric static hand gestures, taken from the American Sign Language. We named our dataset OpenSign and released it to the community for benchmarking. Inception V3 convolutional

Design and Construction of a Cost-Effective Didactic Robotic Arm for Playing Chess, Using an Artificial Vision System

“…The first stage of the corner detection algorithm consists of finding straight lines in the image. For this, the Canny edge detector [40][41][42] and the Hough transform [43][44][45][46] were used, which are available in the OpenCV library.…”

A Deep Learning Framework for Recognizing Both Static and Dynamic Gestures

“…In this work, we employ sign language gestures only as a proof of concept—our static hand gesture detector can be adapted to other classes as well. The static hand gestures detector is detailed in [ 11 ]. For more generic and flexible gesture detection, we propose a multi-stream neural architecture for dynamic gestures recognition, which is integrated with our static hand gestures detector in a unified network.…”

A control scheme for haptic inspection and partial modification of kinematic behaviors

“…Learning from demonstration provides the robots with flexibility and allow them to exploit the workers' experience, as opposed to classical programming which is inflexible, cognitively and physically demanding, timeconsuming and requires technical skills from the humanteacher. The set of motion demonstrations can be either captured via external sensors [1], e.g., a camera, or via internal sensors, i.e., robot's proprioception, during physical human-robot interaction, an approach widely known as kinesthetic teaching [2]- [4]. To encode and generalize the demonstrated kinematic behavior, the most popular approaches involve the utilization of Dynamical Systems (DS) with parameters learned to optimally reflect the set of demonstrated motions.…”

“…LfD can be significantly accelerated if feedback of the current knowledge is provided from the robot-learner to the human-teacher, since it can accelerate the learning process [8]- [10]. Many works propose the bi-directional communication between the human-teacher and the robot-learner [1], [2], [4], [8]- [15]. Some works utilize graphical interfaces to display the path of the robot [12], [13], while others simulate the robot's kinematic behavior [1], [12], [14], [15].…”

“…However, the planar nature of the utilized monitors hinders the user's ability to get a clear understanding of the current knowledge of the robot and navigate in such an environment. A more effective approach involves the autonomous execution of the learned behavior, while allowing physical intervention for modifications [2], [4], [11], [16]. However, in these cases, the user has to wait for the evolution of the dynamical system or synchronize with it and intervene exactly at the specific instance the modification is intended, which can be timeconsuming and cognitive-demanding.…”