Matterport3D: Learning from RGB-D Data in Indoor Environments

Abstract

Access to large, diverse RGB-D datasets is critical for training RGB-D scene understanding algorithms. However, existing datasets still cover only a limited number of views or a restricted scale of spaces. In this paper, we introduce Matterport3D, a large-scale RGB-D dataset containing 10,800 panoramic views from 194,400 RGB-D images of 90 building-scale scenes. Annotations are provided with surface reconstructions, camera poses, and 2D and 3D semantic segmentations. The precise global alignment and comprehensive, diverse panoramic set of views over entire buildings enable a variety of supervised and self-supervised computer vision tasks, including keypoint matching, view overlap prediction, normal prediction from color, semantic segmentation, and region classification.

Optimization of Temporary Debris Management Site Selection and Site Service Regions for Enhancing Postdisaster Debris Removal Operations

“…Recent advances in deep neural networks has significantly improved the performance of object recognition methods. With these developments and the emergence of large data sets such as imagenet (Deng et al., ), scannet (Dai et al., ), SUN3D (Xiao et al., ), and Matterport3D (Chang et al., ), the automated recognition and quantification of debris objects will be feasible in the near future, particularly when both images and 3D data are used to complement each other. This is actually the focus of one of our ongoing research projects.…”

Deep Parametric Indoor Lighting Estimation

“…Legendre et al extend this work to mobile applications and obtain better results by using a collection of videos as training data [14]. Song et al [23] use Matter-port3D [5] dataset and a novel warping procedure in order to support multiple insertion points. We improve on their work by training an end-to-end neural network to predict discrete parametric 3D lights with 3D position, area, color and intensity.…”

Reconstruction of Three-Dimensional (3D) Indoor Interiors with Multiple Stories via Comprehensive Segmentation

“…Dataset-3 was acquired by a Zeb-1 sensor. Dataset-4 and -5 were provided by [20,22] and were captured by RGB-D sensors. Dataset-6 and -7 were acquired by RGB-D sensors from [21].…”

“…The authors would like to gratefully acknowledge Axel Wendt [21] for their help. We would like to thank Angel Chang [22] for their help in accessing and processing the data. We would like to thank Satoshi Ikehata [34] for their help in this paper.…”