Structure guided fusion for depth map inpainting

Qi, Fei; Han, Junyu; Wang, Pengjin; Shi, Guangming; Fu, Li

doi:10.1016/j.patrec.2012.06.003

Cited by 95 publications

(69 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By contrast, we offer improved depth recovery from a single sensing unit (consumer depth camera) without the need for individual per scene optimization, making it highly suitable for mobile sensing applications and dynamic scenes. This extends both the work of [21], which requires additional sensors to achieve the similar results, and the disparity in-painting approach of [17] which does not readily recover transparent and specular object disparity.…”

Section: Discussionmentioning

confidence: 62%

“…Other work has considered depth recovery improvement in depth cameras as a classical image in-painting problem [17]. Qi et al [17] successfully tackles the in-painting of depth shadows due to object occlusion [1] without considering the challenges transparent and specular surfaces.…”

Section: Introductionmentioning

confidence: 99%

“…Qi et al [17] successfully tackles the in-painting of depth shadows due to object occlusion [1] without considering the challenges transparent and specular surfaces. Alternatively [21] augment the existing RGB-D sensing set-up with an additional colour stereo sensor to show improved depth recovery on human faces following fusion method of [23].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improved Depth Recovery In Consumer Depth Cameras via Disparity Space Fusion within Cross-spectral Stereo

Garanderie

Breckon

2014

Proceedings of the British Machine Vision Conference 2014

View full text Add to dashboard Cite

The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. AbstractWe address the issue of improving depth coverage in consumer depth cameras based on the combined use of cross-spectral stereo and near infrared structured light sensing. Specifically we show that fusion of disparity over these modalities prior to subsequent optimization, within the disparity space image, facilitates the recovery of scene depth information in regions where structured light sensing alone fails. This joint approach, leveraging disparity information from both structured light and cross-spectral stereo, facilitates the recovery of global scene depth comprising both texture-less object depth, where stereo sensing commonly fails, and highly reflective object depth, where structured light active sensing commonly fails. The proposed solution is illustrated using dense gradient feature matching and is shown to outperform prior approaches that use late-stage fused cross-spectral stereo depth as a facet of improved sensing for consumer depth cameras.

show abstract

Section: Discussionmentioning

confidence: 62%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved Depth Recovery In Consumer Depth Cameras via Disparity Space Fusion within Cross-spectral Stereo

Garanderie

Breckon

2014

Proceedings of the British Machine Vision Conference 2014

View full text Add to dashboard Cite

show abstract

“…We compared our method to various methods that focus on depth completion for RGB-D images, including Joint bilateral filter (JBF) [14,21], Nonlocal means filter (NLM) [12],Structure-guided fusion (SGF) [20], Spatiotemporal hole filling (SHF) [5], and Guided inpainting and filtering (GIF) [17]. For JBF and SHF, we modified their method to handle single input image.…”

Section: Methodsmentioning

confidence: 99%

“…A variety of joint bilateral filter-based methods have been developed based on this observation to use color images for hole filling [5,20,21]. Median filter has also been extended for depth image completion guided by the color image [18].…”

Section: Related Workmentioning

confidence: 99%

Depth Enhancement via Low-Rank Matrix Completion

Ren

Liu

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

158

132

View full text Add to dashboard Cite

show abstract

Automated wall‐climbing robot for concrete construction inspection

Yang

Feng

et al. 2022

Journal of Field Robotics

View full text Add to dashboard Cite

Human‐made concrete structures require cutting‐edge inspection tools to ensure the quality of the construction to meet the applicable building codes and to maintain the sustainability of the aging infrastructure. This paper introduces a wall‐climbing robot for metric concrete inspection that can reach difficult‐to‐access locations with a close‐up view for visual data collection and real‐time flaws detection and localization. The wall‐climbing robot is able to detect concrete surface flaws (i.e., cracks and spalls) and produce a defect‐highlighted 3D model with extracted location clues and metric measurements. The system encompasses four modules, including a data collection module to capture RGB‐D frames and inertial measurement unit data, a visual–inertial navigation system module to generate pose‐coupled keyframes, a deep neural network module (namely InspectionNet) to classify each pixel into three classes (background, crack, and spall), and a semantic reconstruction module to integrate per‐frame measurement into a global volumetric model with defects highlighted. We found that commercial RGB‐D camera output depth is noisy with holes, and a Gussian‐Bilateral filter for depth completion is introduced to inpaint the depth image. The method achieves the state‐of‐the‐art depth completion accuracy even with large holes. Based on the semantic mesh, we introduce a coherent defect metric evaluation approach to compute the metric measurement of crack and spall area (e.g., length, width, area, and depth). Field experiments on a concrete bridge demonstrate that our wall‐climbing robot is able to operate on a rough surface and can cross over shallow gaps. The robot is capable to detect and measure surface flaws under low illuminated environments and texture‐less environments. Besides the robot system, we create the first publicly accessible concrete structure spalls and cracks data set that includes 820 labeled images and over 10,000 field‐collected images and release it to the research community.

show abstract

Structure guided fusion for depth map inpainting

Cited by 95 publications

References 13 publications

Improved Depth Recovery In Consumer Depth Cameras via Disparity Space Fusion within Cross-spectral Stereo

Improved Depth Recovery In Consumer Depth Cameras via Disparity Space Fusion within Cross-spectral Stereo

Depth Enhancement via Low-Rank Matrix Completion

Automated wall‐climbing robot for concrete construction inspection

Contact Info

Product

Resources

About