Real-Time Plane Segmentation Using RGB-D Cameras

Abstract

Abstract. Real-time 3D perception of the surrounding environment is a crucial precondition for the reliable and safe application of mobile service robots in domestic environments. Using a RGB-D camera, we present a system for acquiring and processing 3D (semantic) information at frame rates of up to 30Hz that allows a mobile robot to reliably detect obstacles and segment graspable objects and supporting surfaces as well as the overall scene geometry. Using integral images, we compute local surface normals. The points are then clustered, segmented, and classified in both normal space and spherical coordinates. The system is tested in different setups in a real household environment. The results show that the system is capable of reliably detecting obstacles at high frame rates, even in case of obstacles that move fast or do not considerably stick out of the ground. The segmentation of all planes in the 3D data even allows for correcting characteristic measurement errors and for reconstructing the original scene geometry in far ranges.

Real-time 3D segmentation of cluttered scenes for robot grasping

“…In [5], [9], [17] table-top scenarios are segmented based on an initial clustering into horizontal support planes. Point clusters supported by these planes, i.e.…”

Fast plane extraction in organized point clouds using agglomerative hierarchical clustering

“…Holz et al [11] first clustered the point cloud in the normal space and further clustered each group by its distance to the origin (0.14s per 640 × 480 points). To avoid per-point normal estimation, Enjarini et al [12] designed the gradient of depth feature for plane segmentation, which could be rapidly computed.…”

“…Compact and semantic modeling of scenes provided by planes is useful in indoor and outdoor 3D reconstruction, visualization, and Building Information Modeling (BIM) [15]. Extracting a major plane is a common strategy for table-top manipulation [11], because it helps segment objects placed on the plane. Planes have been also used for SLAM [16]- [18] and place recognition [19] systems as landmarks, because planes are more robust to noise and more discriminative than points.…”

“…When using a binary heap to find the minimum MSE node, log-linear time complexity O(n log n) can be achieved for this algorithm, where n is the number of points in the sequence. Note that by applying the idea of integral images, as used in [11], [23], merging two nodes and calculating the resulting line fitting MSE become constant time operations.…”

Keypoint-Less, Heuristic Application of Local 3D Descriptors

“…The most common methods to achieve this are based on RANSAC plane detection [24,5,12,17]. However, such methods are not very time-efficient (even in parallel implementations) and the processing times strongly depend on the complexity of the scene.…”