Depth Map Super-Resolution Considering View Synthesis Quality

Lei, Jianjun; Li, Lele; Yue, Huanjing; Wu, Feng; Ling, Nam; Hou, Chunping

doi:10.1109/tip.2017.2656463

Cited by 54 publications

(32 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, global-based depth map SR methods [11,12,13,14,15,4,16] restore the SR image by solving an optimizing problem with certain regression terms. Diebel et al combined LR depth maps with registered HR camera maps to reconstruct HR depth maps, which may cause over-smooth depth map [11].…”

Section: Global-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Joint Residual Pyramid for Depth Map Super-Resolution

Xiao

Cao

Zheng

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

High-resolution depth map can be inferred from a lowresolution one with the guidance of an additional highresolution texture map of the same scene. Recently, deep neural networks with large receptive fields are shown to benefit applications such as image completion. Our insight is that super resolution is similar to image completion, where only parts of the depth values are precisely known. In this paper, we present a joint convolutional neural pyramid model with large receptive fields for joint depth map super-resolution. Our model consists of three sub-networks, two convolutional neural pyramids concatenated by a normal convolutional neural network. The convolutional neural pyramids extract information from large receptive fields of the depth map and guidance map, while the convolutional neural network effectively transfers useful structures of the guidance image to the depth image. Experimental results show that our model outperforms existing state-of-the-art algorithms not only on data pairs of RGB/depth images, but also on other data pairs like color/saliency and color-scribbles/colorized images.

show abstract

Section: Global-based Methodsmentioning

confidence: 99%

“…Ham et al fused appropriate structures of static and dynamic color maps to reconstruct HR depth maps [4]. Lei et al considered view synthesis quality to reconstruct SR depth maps [16].…”

Section: Global-based Methodsmentioning

confidence: 99%

Joint Residual Pyramid for Depth Map Super-Resolution

Xiao

Cao

Zheng

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Yang et al used the joint bilateral filtering on cost values [32] to super-resolve range images and proposed a hierarchical joint bilateral filtering scheme [37] for depth map upsampling. To fully exploit correlation across more domains, the bilateral filter is extended into multi-lateral cases [4], [38], [39]. Min et al [18] have proposed a weighted mode filtering method based on a joint histogram of depth image, where the final solution is determined by seeking a global mode on the histogram.…”

Section: A Depth Image Recoverymentioning

confidence: 99%

Global Auto-Regressive Depth Recovery via Iterative Non-Local Filtering

Yang

Frossard

2019

IEEE Trans. on Broadcast.

View full text Add to dashboard Cite

Existing depth sensing techniques have many shortcomings in terms of resolution, completeness, and accuracy. The performance of 3-D broadcasting systems is therefore limited by the challenges of capturing high-resolution depth data. In this paper, we present a novel framework for obtaining high-quality depth images and multi-view depth videos from simple acquisition systems. We first propose a single depth image recovery algorithm based on auto-regressive (AR) correlations. A fixedpoint iteration algorithm under the global AR modeling is derived to efficiently solve the large-scale quadratic programming. Each iteration is equivalent to a nonlocal filtering process with a residue feedback. Then, we extend our framework to an ARbased multi-view depth video recovery framework, where each depth map is recovered from low-quality measurements with the help of the corresponding color image, depth maps from neighboring views, and depth maps of temporally adjacent frames. AR coefficients on nonlocal spatiotemporal neighborhoods in the algorithm are designed to improve the recovery performance. We further discuss the connections between our model and other methods like graph-based tools, and demonstrate that our algorithms enjoy the advantages of both global and local methods. Experimental results on both the Middleburry datasets and other captured datasets finally show that our method is able to improve the performances of depth images and multi-view depth videos recovery compared with state-of-the-art approaches. Index Terms-Depth recovery, multi-view depth video, autoregressive model, nonlocal correlation, iterative filtering. I. INTRODUCTION 3 D IMAGING applications have rapidly gained interest in recent years, as they promise to enhance the visual experience by extending the conventional 2D video to a more immersive experience. However, the technologies constituting 3D imaging systems are not fully mature yet to

show abstract

“…In addition, driven by computer vision applications, some visual saliency models aim at identifying the salient regions from the image or video [2]. In this survey, we mainly review the latest progress of salient object detection, which has been applied in image/video segmentation [3], [4], image/video retrieval [5], [6], image retargeting [7], [8], image compression [9], image enhancement [10]- [12], video coding [13], foreground annotation [14], quality assessment [15], [16], thumbnail creation [17], action recognition [18], and video summarization [19].…”

mentioning

confidence: 99%

Review of Visual Saliency Detection With Comprehensive Information

Cong

Lei

et al. 2019

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

348

130

View full text Add to dashboard Cite

Visual saliency detection model simulates the human visual system to perceive the scene, and has been widely used in many vision tasks. With the development of acquisition technology, more comprehensive information, such as depth cue, inter-image correspondence, or temporal relationship, is available to extend image saliency detection to RGBD saliency detection, co-saliency detection, or video saliency detection. RGBD saliency detection model focuses on extracting the salient regions from RGBD images by combining the depth information. Co-saliency detection model introduces the inter-image correspondence constraint to discover the common salient object in an image group. The goal of video saliency detection model is to locate the motion-related salient object in video sequences, which considers the motion cue and spatiotemporal constraint jointly. In this paper, we review different types of saliency detection algorithms, summarize the important issues of the existing methods, and discuss the existent problems and future works. Moreover, the evaluation datasets and quantitative measurements are briefly introduced, and the experimental analysis and discission are conducted to provide a holistic overview of different saliency detection methods.

show abstract

Depth Map Super-Resolution Considering View Synthesis Quality

Cited by 54 publications

References 40 publications

Joint Residual Pyramid for Depth Map Super-Resolution

Joint Residual Pyramid for Depth Map Super-Resolution

Global Auto-Regressive Depth Recovery via Iterative Non-Local Filtering

Review of Visual Saliency Detection With Comprehensive Information

Contact Info

Product

Resources

About