Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

Abstract3D shape is a crucial but heavily underutilized cue in today's computer vision systems, mostly due to the lack of a good generic shape representation. With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft Kinect), it is becoming increasingly important to have a powerful 3D shape representation in the loop. Apart from category recognition, recovering full 3D shapes from viewbased 2.5D depth maps is also a critical part of visual understanding. To this end, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the distribution of complex 3D shapes across different object categories and arbitrary poses from raw CAD data, and discovers hierarchical compositional part representations automatically. It naturally supports joint object recognition and shape completion from 2.5D depth maps, and it enables active object recognition through view planning. To train our 3D deep learning model, we construct ModelNet -a large-scale 3D CAD model dataset. Extensive experiments show that our 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.

show abstract

“…Sliding Shapes [30]). Because the depth from these sensors is very reliable, 3D shape can play a more important role in a recognition pipeline.…”

Section: D Shapenetsmentioning

confidence: 99%

3D ShapeNets: A deep representation for volumetric shapes

Song

Khosla

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

1,418

440

View full text Add to dashboard Cite

show abstract

“…On the contrary, our approach is designed towards indoor scenes with no prior training or knowledge about objects we detect in the scenes. Song and Xiao [10], [11] train exemplar-SVMs on synthetic data on hundreds of rendered views. They slide these exemplars in 3D space to search for objects by densely evaluating 3D windows.…”

Section: Related Workmentioning

confidence: 99%

Locating 3D Object Proposals: A Depth-Based Online Approach

Pahwa

Jiang

et al. 2018

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Abstract-2D object proposals, quickly detected regions in an image that likely contain an object of interest, are an effective approach for improving the computational efficiency and accuracy of object detection in color images. In this work, we propose a novel online method that generates 3D object proposals in a RGB-D video sequence. Our main observation is that depth images provide important information about the geometry of the scene. Diverging from the traditional goal of 2D object proposals to provide a high recall, we aim for precise 3D proposals. We leverage on depth information per frame and multi-view scene information to obtain accurate 3D object proposals. Using efficient but robust registration enables us to combine multiple frames of a scene in near real time and generate 3D bounding boxes for potential 3D regions of interest. Using standard metrics, such as Precision-Recall curves and F-measure, we show that the proposed approach is significantly more accurate than the current state-of-the-art techniques. Our online approach can be integrated into SLAM based video processing for quick 3D object localization. Our method takes less than a second in MATLAB on the UW-RGBD scene dataset on a single thread CPU and thus, has potential to be used in low-power chips in Unmanned Aerial Vehicles (UAVs), quadcopters, and drones.

show abstract

“…Other approaches exist which aim to provide 3D completion and classification [35], [34], although only Wu et al [35] uses a single representation for both tasks and is able to do partial-object classification. Amodal perception approaches have also been proposed [28]. While these methods do not provide detailed 3D completions, they do estimate a coarse 3D bounding box from a 2.5D image.…”

Section: D Object Completionmentioning

confidence: 99%

Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception

Burchfiel

Konidaris

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-We introduce Bayesian Eigenobjects (BEOs), a novel object representation that is the first technique able to perform joint classification, pose estimation, and 3D geometric completion on previously unencountered and partially observed query objects. BEOs employ Variational Bayesian Principal Component Analysis (VBPCA) directly on 3D object representations to create generative and compact probabilistic models for classes of 3D objects. Using only depth information, we significantly outperform the current state-of-the-art method for joint classification and 3D completion in both accuracy and query time. Additionally, we show that BEOs are well suited for the extremely challenging task of joint classification, completion, and pose estimation on a large dataset of household objects.

show abstract

Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

Cited by 648 publications

References 26 publications

3D ShapeNets: A deep representation for volumetric shapes

3D ShapeNets: A deep representation for volumetric shapes

Locating 3D Object Proposals: A Depth-Based Online Approach

Bayesian Eigenobjects: A Unified Framework for 3D Robot Perception

Contact Info

Product

Resources

About