Semantic Scene Completion through Multi-Level Feature Fusion

Object-Aware Semantic Scene Completion Through Attention-Based Feature Fusion and Voxel-Points Representation

“…In SSC tasks, all voxels in TSDF do not have the same importance. Invalid voxels located between the camera and the observation surface that can be directly identified as air [34]. Therefore, ASPNet only retains voxels located on the observable surface and behind it, and converts the retained effective voxels into point clouds 𝑃 𝑣𝑎𝑙𝑖𝑑 ∈ 𝑅 𝑁 𝑣𝑎𝑙𝑖𝑑 ×𝐶 𝑣𝑜𝑥𝑒𝑙 in Figure 3.…”

“…Inspired by the Surface-Attention module [34] and ResNet [31] structure, we have designed an adaptive surfaceattention module. This module adaptively transfers local surface features to internal points, thereby supplementing the features of internal "voxel-points".…”

“…For intelligent devices, the ability to infer the complete scene from a single perspective RGB-D image is significant, which can be widely applied in autonomous driving, augmented reality, and robotics technology [1]. Therefore, computer vision researchers are committed to studying how to make machines possess such capabilities.…”

“…3) Our method adopts a main framework based on voxel-points, which effectively improves computational efficiency and reduces computational parameters compared to traditional 3D convolution methods. Compared with the method proposed by TSPNet [34] (our conference paper method), our approach fully utilizes attention mechanisms and combines scene and instance features to achieve accurate reconstruction of indoor scenes. Our proposed method adds object detection and object refinement modules.…”

“…The object refinement module considers the inherent sparsity of voxel space and employs a spatial attention mechanism to focus on effective voxels. Compared with the TSPNet [34] method, the scene reconstruction instance results have more precise geometric details. For SSC mIoU, our network performance improved by 0.2% and 0.2% on the NYU dataset and NYUCAD dataset, respectively.…”