Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Abstract

An intelligent robotic system is one of the key pillars of a smart factory that requires flexibility to handle a variety of tasks. Perception is a key enabling technology for robots. Most existing object detection studies have mainly focused on category-specific objects and have achieved impressive performance. However, robotic systems, particularly in industrial scenarios, typically interact with many category-agnostic objects, which the robot must detect instantly without pre-training. Therefore, in this study, we proposed a template-based detection and segmentation approach, which incorporated a multi-level correlation model and a similarity-refine module, for handling the category-agnostic instance. The proposed approach was then validated and demonstrated in an interactive and adaptive robotic application scenario designed for the typical pick-and-place task. Among them, the picking scan path and location were instructed through human guidance with hand tracking. The neural rendering technology was also introduced to render novel views of the template. The proposed approach was evaluated using a benchmark and verified through a real demonstration.

An interactive gesture control system for collaborative manipulator based on Leap Motion Controller

“…(3) Template matching-based method. [42][43][44][45][46] The principle of the algorithm based on template matching is to compute the similarity between real-time input gesture features and those of the pre-built template library, then choose the type with the highest similarity as the gesture recognition outcome through specific methods. The template matching gesture control is stable, has less computation, and the template library is reduced and modified.…”