Self-Supervised Single-Image Depth Estimation From Focus and Defocus Clues

Lu, Yawen; Milliron, Garrett; Slagter, John; Lu, Guoyu

doi:10.1109/lra.2021.3092258

Cited by 9 publications

(9 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we show qualitative and quantitative results of our proposed framework on a synthetic dataset [15], a real dataset with synthetic defocus blur [8,14] and a real focal stack dataset [22].…”

Section: Methodsmentioning

confidence: 99%

“…[24] further makes use of the implicit mutual information of depth and AIF image to finetune the model. [14] trains a DefocusNet and a FocusNet simultaneously to perform DFD and DFF tasks and the models are supervised by their consistency. [27] add focus volume and differential focus volume to their supervised model to improve estimation quality.…”

Section: Depth From Focus/defocusmentioning

confidence: 99%

“…To evaluate the sharpness of AIF image, we apply a Laplacian filter for the edge map and calculate its variance. By taking the negative log sum of the normalized variance, the blurry estimation loss [14] encourages the image to be as clear as possible:…”

Section: Self-supervised Trainingmentioning

confidence: 99%

“…With the help of deep learning methods, supervised DFD works [15,27] estimate depth in a data-driven way. Recent DFD methods [8,14,24] claim they are freed from being supervised by the depth ground-truth. However, these works utilize the all-in-focus (AIF) image in model training, which is more of a theoretical concept than images captured by camera lens [13] in the real world.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Fully Self-Supervised Depth Estimation from Defocus Clue

Si¹,

Zhao²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Depth-from-defocus (DFD), modeling the relationship between depth and defocus pattern in images, has demonstrated promising performance in depth estimation. Recently, several self-supervised works try to overcome the difficulties in acquiring accurate depth ground-truth. However, they depend on the all-in-focus (AIF) images, which cannot be captured in real-world scenarios. Such limitation discourages the applications of DFD methods. To tackle this issue, we propose a completely self-supervised framework that estimates depth purely from a sparse focal stack. We show that our framework circumvents the needs for the depth and AIF image ground-truth, and receives superior predictions, thus closing the gap between the theoretical success of DFD works and their applications in the real world. In particular, we propose (i) a more realistic setting for DFD tasks, where no depth or AIF image ground-truth is available; (ii) a novel selfsupervision framework that provides reliable predictions of depth and AIF image under the challenging setting. The proposed framework uses a neural model to predict the depth and AIF image, and utilizes an optical model to validate and refine the prediction. We verify our framework on three benchmark datasets with rendered focal stacks and real focal stacks. Qualitative and quantitative evaluations show that our method provides a strong baseline for selfsupervised DFD tasks. The source code is publicly available at https://github.com/Ehzoahis/DEReD.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Depth From Focus/defocusmentioning

confidence: 99%

Section: Self-supervised Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fully Self-Supervised Depth Estimation from Defocus Clue

Si¹,

Zhao²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The authors in [11] jointly explore Conditional Random Field (CRF) and deep CNN for DFD from a single image. The category of self-supervised learning is illustrated by the method outlined in [12], which introduces a framework for depth estimation using the DFD and depth-from-focus algorithms on defocus stacks, or by [3] that uses a Point Spread Function (PSF) convolutional layer to improve depth estimation using the defocus cue.…”

Section: Related Workmentioning

confidence: 99%

2HDED:Net for Joint Depth Estimation and Image Deblurring from a Single Out-of-Focus Image

Nazir

Vaquero

Mucientes

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Depth estimation and all-in-focus image restoration from defocused RGB images are related problems, although most of the existing methods address them separately. The few approaches that solve both problems use a pipeline processing to derive a depth or defocus map as an intermediary product that serves as a support for image deblurring, which remains the primary goal. In this paper, we propose a new Deep Neural Network (DNN) architecture that performs in parallel the tasks of depth estimation and image deblurring, by attaching them the same importance. Our Two-headed Depth Estimation and Deblurring Network (2HDED:NET) is an encoderdecoder network for Depth from Defocus (DFD) that is extended with a deblurring branch, sharing the same encoder. The network is tested on NYU-Depth V2 dataset and compared with several state-of-the-art methods for depth estimation and image deblurring.

show abstract