Spatial Programming for Industrial Robots through Task Demonstration

Abstract

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks.

Augmented reality technology in the manufacturing industry: A review of the last decade

“…Looking at gesture tracking, Kim and Lee (2016) have proposed a method for naturally and directly manipulating 3D AR objects through touch and hand gesture-based interactions in hand-held devices; the touch gesture is used for the AR object selection, whereas the natural hand gesture enables the direct and interactive manipulation of the selected objects. Lambrecht et al (2013) have applied a similar approach, i.e., a combination of marker-less gesture recognition and MAR, to the programing of industrial robots. Caruso et al (2015) have instead proposed an interactive AR system that enables the user to freely interact with virtual objects integrated in a real environment, avoiding the use of cumbersome equipment.…”

Intuitive robot tasks with augmented reality and virtual obstacles

“…A number of groups have applied augmented reality to ease robot programming, including recent contributions by Lambrecht and Krüger [4], [5]. For visualization, most systems use head-mounted displays (HMD) [6], [7], [8] or video see-through (VST) displays [9], [2], [10] or hand-held see-through displays [4], [5].…”

Consensus Tracking by Iterative Learning Control for Linear Heterogeneous Multiagent Systems Based on Fractional-Power Error Signals

“…ILC is used to accomplish control tasks that repeat on finite-time intervals, and it does not require an accurate system model. Due to this attractive feature of ILC, it has been widely used in practical systems and engineering practice, for example, industrial robots that perform repetitive tasks such as welding and handling [9], servo systems whose command signals are periodic functions [10], etc.…”