Adaptive Spotting: Deep Reinforcement Object Search in 3D Point Clouds

Krishna, Onkar; Irie, Go; Wu, Xiaomeng; Kawanishi, Takahito; Kashino, Kunio

doi:10.1007/978-3-030-69535-4_16

Cited by 3 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enable matching with an already existing library object, a distinct description of the objects' geometric representation and matching method between captured and library object is needed. Many works presented in literature rely on the use of handcrafted features and heuristics to match similar object representations (Krishna et al, 2021). This section will discus the two main feature types being Geometry-based and view-based features (Bickel et al, 2022).…”

Section: Feature Extraction and Matchingmentioning

confidence: 99%

Review of Window and Door Type Detection Approaches

Geyter

Bassier²,

Winter³

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. The use of as-built Building Information Models (BIM) has become increasingly commonplace. This process of creating a BIM model from point cloud data, also referred to as Scan-to-BIM, is a mostly manual task. Due to the large amount of manual work, the entire Scan-to-BIM process is time-consuming and error prone. Current research focuses on the automation of the Scan-to-BIM pipeline by applying state-of-the-art techniques on its consecutive steps including the data acquisition, data processing, data interpretation and modelling. By automating the matching and modelling of window and door objects, a considerable amount of time can be saved in the Scan-to-BIM process. This is so because each window and door instance needs to be examined by the modeller and must be adapted to the actual on-site situation. Large object libraries containing predefined window and door objects exists but the matching to the best-fit predefined object remains time consuming. The aim of this research is to examine the possibilities to speed up the modelling of window and door objects. First, a literature review discussing existing methods for window and door detection and matching is presented. Second, the acquired data is examined to explore the capabilities of capturing window and door information for different remote sensing devices. Followed by tests of some commonplace features in the use for window and door occurrence matching and clustering.

show abstract

Section: Feature Extraction and Matchingmentioning

confidence: 99%

Review of Window and Door Type Detection Approaches

Geyter

Bassier²,

Winter³

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Volumetric neural networks are also actively used to work with three-dimensional point clouds [29,30]. In [31] the task of searching for a query object of unknown position and pose in a scene, both given in the form of 3D point cloud data, was studied. This method includes a deep reinforcement learning approach that jointly learns both the features and the efficient search path.This network is successfully trained in an end-to-end manner by integrating a contrastive loss and a reinforcement localization reward.…”

Section: Volumetric Neural Networkmentioning

confidence: 99%

Semantic 3D Reconstruction of a Scene and Its Effective Visualisation

Kniaz

Moshkantsev

Bordodymov

et al. 2022

Proceedings of the 32nd International Conference on Computer Graphics and Vision

View full text Add to dashboard Cite

Single-image 3D scene reconstruction is required in multiple challenging tasks including mobile robotics, industrial monitoring and reconstruction of lost cultural heritage. While modern models demonstrate robust resolution of scene in real time with resolution up to 128 x 128 x 128 voxels, visualization of such detailed of a such detailed voxel model is challenging. A model with 128<sup>3</sup> voxels contains 2097152 simple cubes 16M vertices. It is unfeasible for modern hardware to perform visualization of such voxel models in real-time. Hence a voxel model simplification technique is required to demonstrate reconstruction results in real-time. In this paper, we propose a new algorithm for voxel model simplification using predefined camera views. The algorithm reduces a rigid-body voxel model to a shell voxel model. It keeps only the voxels that are visible from the required view. We demonstrate the effectiveness of the proposed algorithm using a case study with a mobile robot and a state-of-the-art SSZ single-photo 3D reconstruction neural network. We generated a real and a virtual scene with various objects including a statue. We use a mobile robot equipped with a single camera to collect real and synthetic data. We train the SSZ model using the collected data. We developed a dedicated visualization software that implements our algorithm. The comparison of the visualization performance for the full model and its reduced version demonstrates that our algorithm allows to increase the performance by 420 times.

show abstract