Memory‐Efficient Interactive Online Reconstruction From Depth Image Streams

Reichl, Florian; Weiß, Jakob; Westermann, Rüdiger

doi:10.1111/cgf.12779

Cited by 10 publications

(10 citation statements)

References 29 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An embedded processor on an UAV is tightly constrained in processing ability and electrical power availability, and it relies on efficient algorithms to perform any complex task satisfactorily. A voxel-hashing technique is proposed by Nießner et al, 13 which would drastically reduce the graphical processing unit (GPU) memory required to represent 3D point clouds, and Reichl et al 14 build upon this work by processing small batches of imagery in a stream rather than individually, further reducing the amount of GPU memory required for reconstruction.…”

Section: Related Workmentioning

confidence: 99%

Assessment of Structure from Motion for reconnaissance augmentation and bandwidth usage reduction

Roeber

2019

Journal of Defense Modeling & Simulation

View full text Add to dashboard Cite

Modern militaries rely upon remote image sensors for real-time intelligence. A typical remote system consists of an unmanned aerial vehicle, or UAV, with an attached camera. A video stream is sent from the UAV, through a bandwidth-constrained satellite connection, to an intelligence processing unit. In this research, an upgrade to this remote-video-stream method of collection is proposed. A set of synthetic images of a scene captured by an UAV in a virtual environment is sent to a pipeline of computer vision algorithms, collectively known as Structure from Motion. The output of Structure from Motion, a three-dimensional (3D) model, is then assessed in a 3D virtual world as a possible replacement for the images from which it was created. This study shows Structure from Motion results from a modifiable spiral flight path and compares the geoaccuracy of each result. A flattening of height is observed, and an automated compensation for this flattening is proposed and performed. Each reconstruction is also compressed, and the size of the compression is compared with the compressed size of the images from which it was created. A reduction of 49–60% of required space, or bandwidth, is shown. A corresponding video demonstrating this technique is available online.

show abstract

Section: Related Workmentioning

confidence: 99%

Assessment of Structure from Motion for reconnaissance augmentation and bandwidth usage reduction

Roeber

2019

Journal of Defense Modeling & Simulation

View full text Add to dashboard Cite

show abstract

“…This approach is interactive, but not fully real-time. An extremely compact hierarchy is proposed by Reichl et al [RWW16], who only store a binary grid which is updated in a running window fashion.…”

Section: Voxel-based Representationmentioning

confidence: 99%

State of the Art on 3D Reconstruction with RGB‐D Cameras

Zollhöfer

Stotko

Görlitz

et al. 2018

Computer Graphics Forum

256

123

View full text Add to dashboard Cite

The advent of affordable consumer grade RGB‐D cameras has brought about a profound advancement of visual scene reconstruction methods. Both computer graphics and computer vision researchers spend significant effort to develop entirely new algorithms to capture comprehensive shape models of static and dynamic scenes with RGB‐D cameras. This led to significant advances of the state of the art along several dimensions. Some methods achieve very high reconstruction detail, despite limited sensor resolution. Others even achieve real‐time performance, yet possibly at lower quality. New concepts were developed to capture scenes at larger spatial and temporal extent. Other recent algorithms flank shape reconstruction with concurrent material and lighting estimation, even in general scenes and unconstrained conditions. In this state‐of‐the‐art report, we analyze these recent developments in RGB‐D scene reconstruction in detail and review essential related work. We explain, compare, and critically analyze the common underlying algorithmic concepts that enabled these recent advancements. Furthermore, we show how algorithms are designed to best exploit the benefits of RGB‐D data while suppressing their often non‐trivial data distortions. In addition, this report identifies and discusses important open research questions and suggests relevant directions for future work.

show abstract

“…based on moving volume techniques [26,31], representing scenes in terms of blocks of volumes that follow dominant planes [8] or storing TSDF values only near the actual surface areas [1,12,23]. The individual blocks can be managed using tree structures or hash maps as proposed by Nießner et al [23] and respective optimizations [12,13,25]. Furthermore, the replacement of the TSDF representation by a highresolution binary voxel grid has also been considered by Reichl et al [25] to improve the scalability and reduce the memory requirements.…”

Section: Related Workmentioning

confidence: 99%

“…3). Here, the surface is represented in terms of implicit truncated signed distance fields (TSDFs) and stored as a sparse unordered set of voxel blocks using spatial hashing [2,11,12,23,25]. Input to the reconstruction pipeline is an incremental stream of RGB-D images which is processed in an online fashion.…”

Section: Optimization Of the 3d Reconstruction Processmentioning

confidence: 99%

“…tracking [9,12,21,23,25,29,31] (as also used in the SLAMCast system) or using bundle adjustment for globally-consistent reconstruction [2,11]. Using this pose, non-visible voxel block data are streamed out to CPU memory whereas visible blocks in CPU memory are streamed back into GPU memory [12,16,23,25]. In the next step, new voxel blocks are allocated in the volume and the RGB-D data are fused into the volumetric model.…”

Section: Cpu -Gpu Streaming Data Fusion Model Visualizationmentioning

confidence: 99%

See 1 more Smart Citation

Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence

Stotko

Krumpen

Weinmann

et al. 2019

2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

SLAMCast 4 ECs Ours 24 ECsFigure 1: Illustration of our novel highly scalable multi-client live telepresence system. While previous approaches are limited to a low number of up to 4 remote exploration clients, our system is capable of providing an immersive telepresence experience within a live-captured high-quality scene reconstruction to more than 24 clients simultaneously without introducing further latency. ABSTRACTSharing live telepresence experiences for teleconferencing or remote collaboration receives increasing interest with the recent progress in capturing and AR/VR technology. Whereas impressive telepresence systems have been proposed on top of on-the-fly scene capture, data transmission and visualization, these systems are restricted to the immersion of single or up to a low number of users into the respective scenarios. In this paper, we direct our attention on immersing significantly larger groups of people into live-captured scenes as required in education, entertainment or collaboration scenarios. For this purpose, rather than abandoning previous approaches, we present a range of optimizations of the involved reconstruction and streaming components that allow the immersion of a group of more than 24 users within the same scene -which is about a factor of 6 higher than in previous work -without introducing further latency or changing the involved consumer hardware setup. We demonstrate that our optimized system is capable of generating high-quality scene reconstructions as well as providing an immersive viewing experience to a large group of people within these live-captured scenes.

show abstract

Memory‐Efficient Interactive Online Reconstruction From Depth Image Streams

Cited by 10 publications

References 29 publications

Assessment of Structure from Motion for reconnaissance augmentation and bandwidth usage reduction

Assessment of Structure from Motion for reconnaissance augmentation and bandwidth usage reduction

State of the Art on 3D Reconstruction with RGB‐D Cameras

Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence

Contact Info

Product

Resources

About