Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Meerits, Siim; Nozick, Vincent; Saitō, Hideo

doi:10.1007/s11554-017-0736-x

Cited by 9 publications

(13 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compared our 3D reconstruction method with another MLS method that uses mesh zippering [24] and with Poisson surface reconstruction [19]. Some results for highly dynamic and large scenes are shown in Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Fig. 11 Reconstructing a fast ball throw in a large scene with five RGB-D cameras: (a) shows our method together with camera positions, (b) uses MLS-based mesh zippering [24], and (c) shows Poisson reconstruction method [19]. Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Fig. 12 Comparison of 3D reconstruction methods: (a) shows our method, (b) uses MLS-based mesh zippering [24], and (c) shows the Poisson reconstruction method [19]. Zippering shows bad edge quality and occasionally has incorrect triangles protruding from surfaces.…”

Section: Resultsmentioning

confidence: 99%

“…In applications, however, triangle mesh models are preferred over points. Meerits et al [24] used a method inspired by mesh zippering [25] to generate triangle meshes for MLS surfaces. The drawback of this work is that when using many cameras, mesh density becomes unnecessarily high and the mesh joints can present some ambiguities.…”

Section: Frame-based Methodsmentioning

confidence: 99%

See 3 more Smart Citations

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

et al. 2018

Self Cite

View full text Add to dashboard Cite

Multi-view dynamic three-dimensional reconstruction has typically required the use of custom shutter-synchronized camera rigs in order to capture scenes containing rapid movements or complex topology changes. In this paper, we demonstrate that multiple unsynchronized low-cost RGB-D cameras can be used for the same purpose. To alleviate issues caused by unsynchronized shutters, we propose a novel depth frame interpolation technique that allows synchronized data capture from highly dynamic 3D scenes. To manage the resulting huge number of input depth images, we also introduce an efficient moving least squares-based volumetric reconstruction method that generates triangle meshes of the scene. Our approach does not store the reconstruction volume in memory, making it memory-efficient and scalable to large scenes. Our implementation is completely GPU based and works in real time. The results shown herein, obtained with real data, demonstrate the effectiveness of our proposed method and its advantages compared to stateof-the-art approaches.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Frame-based Methodsmentioning

confidence: 99%

See 2 more Smart Citations

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…The 3D model can also be used to create a textured 3D mesh, or to anchor and orient the augmented rendering overlays into the presented view. More recently, free viewpoint rendering is converging with mesh-based rendering for enabling real-time applications with multiple sensors as scene data sources, as seen in [20,21].…”

Section: B View Renderingmentioning

confidence: 99%

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

Dima

Sjöström

2021

IEEE Access

View full text Add to dashboard Cite

Remote operation of diggers, scalers, and other tunnel-boring machines has significant benefits for worker safety in underground mining. Real-time augmentation of the presented remote views can further improve the operator effectiveness through a more complete presentation of relevant sections of the remote location. In safety-critical applications, such augmentation cannot depend on preconditioned data, nor generate plausible-looking yet inaccurate sections of the view. In this paper, we present a capture and rendering pipeline for real time view augmentation and novel view synthesis that depends only on the inbound data from lidar and camera sensors. We suggest an on-the-fly lidar filtering for reducing point oscillation at no performance cost, and a full rendering process based on lidar depth upscaling and in-view occluder removal from the presented scene. Performance assessments show that the proposed solution is feasible for real-time applications, where per-frame processing fits within the constraints set by the inbound sensor data and within framerate tolerances for enabling effective remote operation.

show abstract

MARS: parallelism-based metrically accurate 3D reconstruction system in real-time

Zhang

Wang

et al. 2020

J Real-Time Image Proc

View full text Add to dashboard Cite

Due to the increasing application demands, lightweight device-based 3D recovery draws many attentions from a wide group of researchers in both academic and industrial fields. The current 3D reconstruction solutions are commonly achieved either using depth data or RGB data. The depth data usually come from a deliberately designed hardware for specific tasks, while the RGB data-based solutions only employ a single RGB camera with vision-based computing algorithms. Limitations are expected from both. Depth sensors are commonly either bulky or relatively expensive compared to RGB cameras, thus of less flexibility. Normal RGB cameras usually have better mobility but less accuracy in 3D sensing than depth sensors. Recently, machine learning based depth estimation has also been presented. However, its accuracy are still limited. To improve the flexibility of the 3D reconstruction system without loss in accuracy, this paper presents a solution of unconstrained Metrically Accurate 3D Reconstruction System (MARS) for 3D sensing based on a consumer-grade camera. With a simple initialization from a depth map, the system can achieve incremental 3D reconstruction with a stable metric scale. Experiments are conducted using both real-world data and public datasets. Competitive results are obtained using the proposed system compared with several existing methods.

show abstract

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Cited by 9 publications

References 49 publications

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

MARS: parallelism-based metrically accurate 3D reconstruction system in real-time

Contact Info

Product

Resources

About