Discrete Cosine Transform Network for Guided Depth Map Super-Resolution

Zhao, Zixiang; Zhang, Jiangshe; Xu, Shuang; Lin, Zudi; Pfister, Hanspeter

doi:10.1109/cvpr52688.2022.00561

Cited by 57 publications

(23 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, high-resolution and high-quality RGB images are usually available on depth sensors. Various guided depth map SR methods [54], [55], [56], [57], [58] have been proposed to fuse the details of RGB images to the depth maps for the same captured scenes and thus improve the resolution of the depth maps. Traditional guided depth SR methods [54], [55] use an energy function based on different priors and regularization terms to find the optimized SR depth.…”

Section: Guided Depth Super-resolution Methodsmentioning

confidence: 99%

“…Traditional guided depth SR methods [54], [55] use an energy function based on different priors and regularization terms to find the optimized SR depth. With deep learning techniques becoming popular in computer vision tasks, many deep learning-based guided depth SR methods [56], [57], [58] have been proposed to establish the mapping relationship between the LR depth map with the corresponding high-resolution RGB image and the SR depth map. These methods outperform traditional methods.…”

Section: Guided Depth Super-resolution Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Super-Resolution Phase Retrieval Network for Single-Pattern Structured Light 3D Imaging

Song

Liu

Sowmya

et al. 2023

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Structured light 3D imaging is often used for obtaining accurate 3D information via phase retrieval. Single-pattern structured light 3D imaging is much faster than multi-pattern versions. Current phase retrieval methods for single-pattern structured light 3D imaging are however not accurate enough. Besides, the projector resolution in a structured light 3D imaging system is expensive to improve due to hardware costs. To address the issues of low accuracy and low resolution of single-pattern structured light 3D imaging, this work proposes a super-resolution phase retrieval network (SRPRNet). Specifically, a phase-shifting module is proposed to extract multi-scale features with different phase shifts, and a refinement and super-resolution module is proposed to obtain refined and super-resolution phase components. After phase demodulation and unwrapping, high-resolution absolute phase is obtained. A sine shifting loss and a cosine shifting loss are also introduced to form the regularization term of the loss function. As far as can be ascertained, the proposed SRPRNet is the first network for super-resolution phase retrieval by using a single pattern, and it can also be used for standard-resolution phase retrieval. Experimental results on three datasets show that SRPRNet achieves state-of-the-art performance on 1×, 2×, and 4× superresolution phase retrieval tasks.

show abstract

Section: Guided Depth Super-resolution Methodsmentioning

confidence: 99%

Section: Guided Depth Super-resolution Methodsmentioning

confidence: 99%

Super-Resolution Phase Retrieval Network for Single-Pattern Structured Light 3D Imaging

Song

Liu

Sowmya

et al. 2023

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…Table2, our method achieves the lowest RMSE in noisy RGB-D-D dataset and the lowest RMSE in ToFMark dataset, which proves its ability for noise removing. As shown in Fig.6, it is observed that DKN (Kim, Ponce, and Ham 2021) and DCTNet(Zhao et al 2022) introduce some texture artifacts and noise in the low-frequency region, while SFG recovers clean surface owing to PEA with effective texture removing.…”

mentioning

confidence: 92%

“…In order to balance the training time and network performance, the parameters L, L , K, T are set to 3, 6, 3, 2 in this paper. We quantitatively and visually compare our method with 13 state-of-the-art (SOTA) methods: TGV (Ferstl et al 2013), FBS (Barron and Poole 2016), MSG (Tak-Wai, Loy, and Tang 2016), DJF (Li et al 2016), DJFR (Li et al 2019), GbFT (AlBahar and Huang 2019), PAC (Su et al 2019), CUNet (Deng and Dragotti 2020), FDKN (Kim, Ponce, and Ham 2021), DKN (Kim, Ponce, and Ham 2021), FDSR (He et al 2021), CTKT (Sun et al 2021) and DCTNet(Zhao et al 2022). For simplicity, we name our Structure Flow-Guided method as SFG.…”

mentioning

confidence: 99%

Structure Flow-Guided Network for Real Depth Super-Resolution

Yuan¹,

Jiang²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent advances of deep learning algorithms have revolutionized the area of computer vision; tremendous progress has been achieved in a variety of domains, such as image classification [27,46], object localization [131,147] and semantic segmentation [108,151]. GDSR is no exception: a myriad of neural networks have been developed and advanced the state of the art [17,154,156]. The learning-based method is suitable for a scene where there is a large amount of training data.…”

Section: Introductionmentioning

confidence: 99%

Guided Depth Map Super-resolution: A Survey

Zhang¹,

Liu²,

Jiang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Guided depth map super-resolution (GDSR), which aims to reconstruct a high-resolution (HR) depth map from a low-resolution (LR) observation with the help of a paired HR color image, is a longstanding and fundamental problem, it has attracted considerable attention from computer vision and image processing communities. A myriad of novel and effective approaches have been proposed recently, especially with powerful deep learning techniques. This survey is an effort to present a comprehensive survey of recent progress in GDSR. We start by summarizing the problem of GDSR and explaining why it is challenging. Next, we introduce some commonly used datasets and image quality assessment methods. In addition, we roughly classify existing GDSR methods into three categories, i.e., filtering-based methods, prior-based methods, and learning-based methods. In each category, we introduce the general description of the published algorithms and design principles, summarize the representative methods, and discuss their highlights and limitations. Moreover, the depth related applications are introduced. Furthermore, we conduct experiments to evaluate the performance of some representative methods based on unified experimental configurations, so as to offer a systematic and fair performance evaluation to readers. Finally, we conclude this survey with possible directions and open problems for further research. All the related materials can be found at https://github.com/zhwzhong/Guided-Depth-Map-Super-resolution-A-Survey.CCS Concepts: • General and reference → Surveys and overviews.

show abstract

Discrete Cosine Transform Network for Guided Depth Map Super-Resolution

Cited by 57 publications

References 43 publications

Super-Resolution Phase Retrieval Network for Single-Pattern Structured Light 3D Imaging

Super-Resolution Phase Retrieval Network for Single-Pattern Structured Light 3D Imaging

Structure Flow-Guided Network for Real Depth Super-Resolution

Guided Depth Map Super-resolution: A Survey

Contact Info

Product

Resources

About