A new method for inpainting of depth maps from time-of-flight sensors based on a modified closing by reconstruction algorithm

Garduño-Ramón, Marco Antonio; Terol-Villalobos, Iván R.; Osornio-Ríos, Roque Alfredo; Morales-Hernandez, Luis A.

doi:10.1016/j.jvcir.2017.05.003

Cited by 12 publications

(10 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…8 and Fig. 9) and quantitatively (Table 2) demonstrate that our depth fusion algorithm outperforms the inpainting methods [12], [13], [62].…”

Section: A Evaluation Of the Depth Refinement Resultsmentioning

confidence: 82%

“…Many heuristic methods that are based on image filtering or optimization have been proposed for filling the holes in depth maps. Most of these methods [11], [12] focus on the depth maps that are generated by depth sensors such as the Microsoft Kinect. These depth maps are typically more accurate and complete than depth maps that are obtained via MVS.…”

Section: B Depth Estimationmentioning

confidence: 99%

“…These depth maps are typically more accurate and complete than depth maps that are obtained via MVS. Modified closing by reconstruction (McBR) [12] improved the depth maps from time-of-flight sensors by using a modified morphological closing filter. Several reconstruction iterations are required for the removal of small holes from the depth maps; however, this approach is ineffective for large holes in outdoor scenes with structures that are completely missing.…”

Section: B Depth Estimationmentioning

confidence: 99%

“…Besides the depth refinement methods tailored for IBR problem, there are a large number of works, which formulate the depth refinement task as depth inpainting [11], [12] and depth completion [13], [14]. The achievements of deep neural networks in recent years have encouraged researchers to implicitly model problems using many training examples.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth

Liu

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Image-based rendering (IBR) attempts to synthesize novel views using a set of observed images. Some IBR approaches (such as light fields) have yielded impressive high-quality results on smallscale scenes with dense photo capture. However, available wide-baseline IBR methods are still restricted by the low geometric accuracy and completeness of multi-view stereo (MVS) reconstruction on low-textured and non-Lambertian surfaces. The issues become more significant in large-scale outdoor scenes due to challenging scene content, e.g., buildings, trees, and sky. To address these problems, we present a novel IBR algorithm that consists of two key components. First, we propose a novel depth refinement method that combines MVS depth maps with monocular depth maps predicted via deep learning. A lookup table remap is proposed for converting the scale of the monocular depths to be consistent with the scale of the MVS depths. Then, the rescaled monocular depth is used as the constraint in the minimum spanning tree (MST)based nonlocal filter to refine the per-view MVS depth. Second, we present an efficient shape-preserving warping algorithm that uses superpixels to generate the warped images and blend expected novel views of scenes. The proposed method has been evaluated on public MVS and view synthesis datasets, as well as newly captured large-scale outdoor datasets. In comparison with state-of-the-art methods, the experimental results demonstrated that the proposed method can obtain more complete and reliable depth maps for the challenging large-scale outdoor scenes, thereby resulting in more promising novel view synthesis. INDEX TERMS Image-based rendering, multi-view stereo, monocular depth estimation, view synthesis, outdoor scenes.

show abstract

“…8 and Fig. 9) and quantitatively (Table 2) demonstrate that our depth fusion algorithm outperforms the inpainting methods [12], [13], [62].…”

Section: A Evaluation Of the Depth Refinement Resultsmentioning

confidence: 82%

Section: B Depth Estimationmentioning

confidence: 99%

Section: B Depth Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth

Liu

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Each type of noise corresponds to specific theoretical or empirical noise models. Most of 3D/depth image improvement methods mainly focus on reducing spatial axial and lateral noise, smoothing temporal fluctuations and filling non-measured pixels [11].…”

Section: Related Workmentioning

confidence: 99%

Dynamic 3D reconstruction improvement via intensity video guided 4D fusion

Zhang

Maniatis

Horna

et al. 2018

Journal of Visual Communication and Image Representation

View full text Add to dashboard Cite

The availability of high-speed 3D video sensors has greatly facilitated 3D shape acquisition of dynamic and deformable objects, but high frame rate 3D reconstruction is always degraded by spatial noise and temporal fluctuations. This paper presents a simple yet powerful dynamic 3D reconstruction improvement algorithm based on intensity video guided multi-frame 4D fusion. Temporal tracking of intensity image points (of moving and deforming objects) allows registration of the corresponding 3D model points, whose 3D noise and fluctuations are then reduced by spatio-temporal multi-frame 4D fusion. We conducted simulated noise tests and real experiments on four 3D objects using a 1000 fps 3D video sensor. The results demonstrate that the proposed algorithm is effective at reducing 3D noise and is robust against intensity noise. It outperforms existing algorithms with good scalability on both stationary and dynamic objects.

show abstract

Multi-objective Cartesian Genetic Programming optimization of morphological filters in navigation systems for Visually Impaired People

Dourado

Pedrino

2020

Applied Soft Computing

View full text Add to dashboard Cite

A new method for inpainting of depth maps from time-of-flight sensors based on a modified closing by reconstruction algorithm

Cited by 12 publications

References 47 publications

Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth

Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth

Dynamic 3D reconstruction improvement via intensity video guided 4D fusion

Multi-objective Cartesian Genetic Programming optimization of morphological filters in navigation systems for Visually Impaired People

Contact Info

Product

Resources

About