Construction of a 3D object recognition and manipulation database from grasp demonstrations

Abstract

Object recognition and manipulation are critical for enabling robots to operate within a household environment. There are many grasp planners that can estimate grasps based on object shape, but these approaches often perform poorly because they miss key information about non-visual object characteristics, such as weight distribution, fragility of materials, and usability characteristics. Object model databases can account for this information, but existing methods for constructing 3D object recognition databases are time and resource intensive, often requiring specialized equipment, and are therefore dicult to apply to robots in the eld. We present an easy-to-use system for constructing object models for 3D object recognition and manipulation made possible by advances in web robotics. The database consists of point clouds generated using a novel iterative point cloud registration algorithm, which includes the encoding of manipulation data and usability characteristics. The system requires no additional equipment other than the robot itself, and non-expert users can demonstrate grasps through an intuitive web interface with virtually no training required. We validate the system with data collected from both a crowdsourcing user study and a set of grasps demonstrated by an expert user. We show that the crowdsourced grasps can produce successful autonomous grasps, and furthermore the demonstration approach outperforms purely vision-based grasp planning approaches for a wide variety of object classes.

A Comparison of Remote Robot Teleoperation Interfaces for General Object Manipulation

“…The first condition uses the free positioning approach, described in Section 3.1. As such, it is very similar to our previous remote pick-and-place interface [11], containing a 3D viewer side-by-side with the camera stream. Along with the ring-and-arrow marker, the 3D viewer includes renderings of point clouds from the two depth cameras and a model of the robot in its current pose, to provide context and increase the user's situational awareness.…”

“…Next, the algorithm performs a few calculations required to compute the heuristics, including cropping the point cloud to the workspace, segmenting the dominant plane, computing its surface normal, and separating the cropped point cloud into local geometry clusters based on Euclidean distance (lines 4-7). For each grasp, the algorithm computes a ranking based on our three heuristics (lines [10][11][12][13][14]. The final ranking is determined from a linear combination of the three heuristics, which can be assigned different priority by adjusting the constants α and β (line 14).…”

“…The ring-and-arrow marker has also found use in web interfaces. In previous work, we used the marker in an interface for crowdsourcing object recognition and picking [11]. More recently, Grice and Kemp showed similar controls superimposed on a camera stream displayed in a web interface, rather than rendering them in a 3D environment [10].…”

Leveraging depth data in remote robot teleoperation interfaces for general object manipulation

“…The ring-and-arrow marker has also found use in web interfaces. In previous work (Kent et al, 2016), we used the marker in an interface for crowdsourcing object recognition and picking. More recently, Kemp and Grice (2016) showed similar controls superimposed on a camera stream displayed as an overlay on a web interface, rather than rendering them in a 3D environment.…”

Advances in sensing and processing methods for three-dimensional robot vision

“…Visual servo system is used for robot manipulator, 77 localization of humanoid robot for locomotion tasks, 78 object recognition with pose estimation, 79,80 robot ego-motion estimation, 81,82 and object grasping. 83,84 Aeronautics and astronautics. Satellite remote sensing image information is usually very huge.…”