Learning local depth regression from defocus blur by soft-assignment encoding

Abstract

We present a novel, to the best of our knowledge, patch-based approach for depth regression from defocus blur. Most state-of-the-art methods for depth from defocus (DFD) use a patch classification approach among a set of potential defocus blurs related to a depth, which induces errors due to the continuous variation of the depth. Here, we propose to adapt a simple classification model using a soft-assignment encoding of the true depth into a membership probability vector during training and a regression scale to predict intermediate depth values. Our method uses no blur model or scene model; it only requires a training dataset of image patches (either raw, gray scale, or RGB) and their corresponding depth label. We show that our method outperforms both classification and direct regression on simulated images from structured or natural texture datasets, and on raw real data having optical aberrations from an active DFD experiment.

Co-design of an active depth from defocus camera for surface inspection

“…Experimental validations have been conducted on real objects, including metallic parts, with a reference depth map from an active stereoscopic camera. Improvement of the accuracy of our ADFD system could be obtained using a different image processing for the depth estimation such as the work of Leroy et al, 6 using deep learning for depth regression from a blurred patch. Further works will concern application of this technique to data provided by our optimized system.…”

“…To solve this problem, depth estimation methods have been proposed, either based on statistical scene model and blur calibration, 2,5 or on direct depth estimation from a blurred patch using deep learning. 6 However, passive DFD methods rely on a large amount of texture on the scene, which is not the case of industrial metallic parts. Hence, active DFD estimation techniques, based on the projection of patterns on the object have been proposed in the literature.…”

Multitask deep co-design for extended depth of field and depth from defocus

“…Indeed, EDOF can benefit from high-frequency transfer, 24 while DfD can use the chromatic shift (distance between focal plane of different wavelengths) to disambiguate relative depth to any focal point. 25 For this reason, we choose a chromatic Cooke triplet as our lens design and we optimize three main parameters, namely, the curvature radius of the first lens surface, the aperture size and the distance of the sensor to the last surface, which respectively account for the chromatic properties, the depth of field and the focal plane distance of the optical system. The initial optics is set with a focal length of 26.8mm, an aperture of 4.68mm and a focus distance of 325mm at a wavelength of 530nm.…”

3D positioning and autofocus of the particle field based on the depth-from-defocus method and the deep networks

“…Continuous depths can then be calculated by taking the weighted average of all possible depths with their probabilities. While Leroy et al proposed a regression method to address the discretization of depth values [18], the adjacent particles' overlaps may still affect the probability distribution, and achieving accurate and consecutive depth estimation remains an ongoing research challenge.…”

“…Sachs et al have presented deterministic algorithms and deep neural networks that can recognize the size of up to four particle species simultaneously, with a particle diameter ranging from 1.14 µm to 5.03 µm [17]. Leroy et al used both the soft-assignment encoding and the DfD method to determine the intermediate depth for a single object from defocus blur images [18]. We have combined the DfD method with an efficient CNN called EfficientNet, which has demonstrated exceptional performance in image classification and object detection tasks [19].…”