Anisotropic Convolutional Networks for 3D Semantic Scene Completion

Li, Jie; Han, Kai; Wang, Peng; Liu, Yu; Yuan, Xia

doi:10.1109/cvpr42600.2020.00341

Cited by 53 publications

(64 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate MonoScene on popular real-world SSC datasets being, indoor NYUv2 [58] and outdoor Se-manticKitti [3]. Because we first address 3D SSC from a 2D image, we detail our non-trivial adaptation of recent SSC baselines [9,39,55,69] (Sec. 4.1) and then detail our performance (Sec.…”

Section: Methodsmentioning

confidence: 99%

“…The task gained attention lately, and is thoroughly reviewed in a survey [56]. Existing works all use geometrical inputs like depth [12,25,[39][40][41][42]45], occupancy grids [13,25,55,69] or point cloud [53,81]. Truncated Signed Distance Function (TSDF) were also proved informative [6,9,10,12,20,21,41,59,64,77,79].…”

Section: Related Workmentioning

confidence: 99%

“…Among others originalities, some SSC works use adversarial training to guide realism [10,64], exploit multi-task [6,38], or use lightweight networks [40,55]. Of interest for us, while others have used RGB as input [6,8,9,14,20,20,25,29,39,40,42,45,81] it is always along other geometrical input (e.g. depth, TSDF, etc.).…”

Section: Related Workmentioning

confidence: 99%

“…Contextual features are crucial for semantics [71] and SSC [56] tasks. A simple strategy is to concatenate multiscale features with skip connections [9,19,45,59,77] or use dilated convolutions for large receptive fields [73], for example with the popular Atrous Spatial Pyramid Pooling (ASPP) [7] also used in SSC [39,40,45,55]. Long-range information is gathered by self-attention in [24,33] and global pooling in [72,76].…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

MonoScene: Monocular 3D Semantic Scene Completion

Cao,

de Charette

2021

Preprint

View full text Add to dashboard Cite

MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the dense geometry and semantics of a scene are inferred from a single monocular RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we solve the complex problem of 2D to 3D scene reconstruction while jointly inferring its semantics. Our framework relies on successive 2D and 3D UNets bridged by a novel 2D-3D features projection inspiring from optics and introduces a 3D context relation prior to enforce spatio-semantic consistency. Along with architectural contributions, we introduce novel global scene and local frustums losses. Experiments show we outperform the literature on all metrics and datasets while hallucinating plausible scenery even beyond the camera field of view. Our code and trained models are available at https://github.com/cv-rits/MonoScene

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

MonoScene: Monocular 3D Semantic Scene Completion

Cao,

de Charette

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They discretize incomplete scans into voxel or TSDF grids and predict per-voxel occupancy with 3D CNNs for scene completion. With the fully convolution design, it also supports scene understanding tasks joint with completion, such as semantic completion or segmentation [55,65,34,39,61,58,33,36,9,25,13]. However, the expensive 3D CNNs consume much computation that hinders them with limited resolution, which appears more obvious on instance objects.…”

Section: Related Workmentioning

confidence: 99%

RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

Nie

Hou

Han

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-Modal Distillation

Vobecký

Hurych

Siméoni

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We describe an approach to predict open-vocabulary 3D semantic voxel occupancy map from input 2D images with the objective of enabling 3D grounding, segmentation and retrieval of free-form language queries. This is a challenging problem because of the 2D-3D ambiguity and the open-vocabulary nature of the target tasks, where obtaining annotated training data in 3D is difficult. The contributions of this work are three-fold. First, we design a new model architecture for open-vocabulary 3D semantic occupancy prediction. The architecture consists of a 2D-3D encoder together with occupancy prediction and 3D-language heads. The output is a dense voxel map of 3D grounded language embeddings enabling a range of open-vocabulary tasks. Second, we develop a tri-modal self-supervised learning algorithm that leverages three modalities: (i) images, (ii) language and (iii) LiDAR point clouds, and enables training the proposed architecture using a strong pre-trained vision-language model without the need for any 3D manual language annotations. Finally, we demonstrate quantitatively the strengths of the proposed model on several open-vocabulary tasks: Zero-shot 3D semantic segmentation using existing datasets; 3D grounding and retrieval of free-form language queries, using a small dataset that we propose as an extension of nuScenes. You can find the project page here https://vobecant.github.io/POP3D.

show abstract

Anisotropic Convolutional Networks for 3D Semantic Scene Completion

Cited by 53 publications

References 16 publications

MonoScene: Monocular 3D Semantic Scene Completion

MonoScene: Monocular 3D Semantic Scene Completion

RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-Modal Distillation

Contact Info

Product

Resources

About