Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction

Shi, Yifei; Chang, Angel X.; Wu, Zhelun; Savva, Manolis; Xu, Kai

doi:10.1109/cvpr.2019.00187

Cited by 27 publications

(20 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work in [38] achieves reasonable results on instance segmentation of 3D point clouds via analyzing point patch context. In [39], a recursive auto-encoder based approach is proposed to predict 3D object detection via exploring hierarchical context priors in 3D object layout. Inspired by the self-attention idea in natural language processing [40], recent works connect the self-attention mechanism with contextual information mining to improve scene understanding tasks such as image recognition [41], semantic segmentation [11] and point cloud recognition [42].…”

Section: Contextual Informationmentioning

confidence: 99%

MLCVNet: Multi-Level Context VoteNet for 3D Object Detection

Xie

Lai

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

139

154

View full text Add to dashboard Cite

In this paper, we address the 3D object detection task by capturing multi-level contextual information with the selfattention mechanism and multi-scale feature fusion. Most existing 3D object detection methods recognize objects individually, without giving any consideration on contextual information between these objects. Comparatively, we propose Multi-Level Context VoteNet (MLCVNet) to recognize 3D objects correlatively, building on the state-of-the-art VoteNet. We introduce three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels. Specifically, a Patch-to-Patch Context (PPC) module is employed to capture contextual information between the point patches, before voting for their corresponding object centroid points. Subsequently, an Object-to-Object Context (OOC) module is incorporated before the proposal and classification stage, to capture the contextual information between object candidates. Finally, a Global Scene Context (GSC) module is designed to learn the global scene context. We demonstrate these by capturing contextual information at patch, object and scene levels. Our method is an effective way to promote detection accuracy, achieving new state-of-the-art detection performance on challenging 3D object detection datasets, i.e., SUN RGBD and ScanNet. We also release our code at https://github.com/NUAAXQ/MLCVNet.

show abstract

Section: Contextual Informationmentioning

confidence: 99%

MLCVNet: Multi-Level Context VoteNet for 3D Object Detection

Xie

Lai

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

139

154

View full text Add to dashboard Cite

show abstract

“…Zhao et al (2011) use a stochastic grammar with three production rules: AND, OR and SET to model the scene layout, detected objects, planes and the background. Scene synthesis methods that aim to generate realistic scene models and layouts incorporate knowledge about an objects' context and the scene composition either directly or indirectly (Jiang et al 2018;Shi et al 2019). Jiang et al (2018) describe a configurable 3D scene synthesis pipeline based on stochastic grammars, so-called spatial and-or graphs.…”

Section: D Object Context and Scene Layoutmentioning

confidence: 99%

“…GRAINS Li et al (2018a) combine a recursive VAE with object retrieval to iteratively generate a layout and objects. Shi et al (2019) also suggests an iterative approach based on a novel variational recursive autoencoder. Kulkarni et al (2019) on the other hand, create a 3D scene given a 2D image.…”

Section: D Object Context and Scene Layoutmentioning

confidence: 99%

Learning 3D Semantic Scene Graphs with Instance Embeddings

2022

View full text Add to dashboard Cite

A 3D scene is more than the geometry and classes of the objects it comprises. An essential aspect beyond object-level perception is the scene context, described as a dense semantic network of interconnected nodes. Scene graphs have become a common representation to encode the semantic richness of images, where nodes in the graph are object entities connected by edges, so-called relationships. Such graphs have been shown to be useful in achieving state-of-the-art performance in image captioning, visual question answering and image generation or editing. While scene graph prediction methods so far focused on images, we propose instead a novel neural network architecture for 3D data, where the aim is to learn to regress semantic graphs from a given 3D scene. With this work, we go beyond object-level perception, by exploring relations between object entities. Our method learns instance embeddings alongside a scene segmentation and is able to predict semantics for object nodes and edges. We leverage 3DSSG, a large scale dataset based on 3RScan that features scene graphs of changing 3D scenes. Finally, we show the effectiveness of graphs as an intermediate representation on a retrieval task.

show abstract

“…In addition, PointFusion [43] introduced a novel framework, in which the image data and the raw point cloud data are independently processed by a CNN (Convolutional Neural Network) and a PointNet architecture respectively, followed by a fusion network combining their output results. Instead of utilizing both 2D and 3D information, [35] took 3D point data only, and utilized the geometric and hierarchical contextual information for 3D object detection. Recently, with only 3D input, VoteNet [28] introduced a deep learning-based Hough voting strategy for 3D object detection from point clouds.…”

Section: Related Workmentioning

confidence: 99%

VENet: Voting Enhancement Network for 3D Object Detection

Xie

Lai

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Hough voting, as has been demonstrated in VoteNet, is effective for 3D object detection, where voting is a key step. In this paper, we propose a novel VoteNet-based 3D detector with vote enhancement to improve the detection accuracy in cluttered indoor scenes. It addresses the limitations of current voting schemes, i.e., votes from neighboring objects and background have significant negative impacts. Before voting, we replace the classic MLP with the proposed Attentive MLP (AMLP) in the backbone network to get better feature description of seed points. During voting, we design a new vote attraction loss (VALoss) to enforce vote centers to locate closely and compactly to the corresponding object centers. After voting, we then devise a vote weighting module to integrate the foreground/background prediction into the vote aggregation process to enhance the capability of the original VoteNet to handle noise from background voting. The three proposed strategies all contribute to more effective voting and improved performance, resulting in a novel 3D object detector, termed VENet. Experiments show that our method outperforms state-of-the-art methods on benchmark datasets. Ablation studies demonstrate the effectiveness of the proposed components.

show abstract

Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction

Cited by 27 publications

References 60 publications

MLCVNet: Multi-Level Context VoteNet for 3D Object Detection

MLCVNet: Multi-Level Context VoteNet for 3D Object Detection

Learning 3D Semantic Scene Graphs with Instance Embeddings

VENet: Voting Enhancement Network for 3D Object Detection

Contact Info

Product

Resources

About