A simplified approach using deep neural network for fast and accurate shape from focus

Abstract

Three-dimensional shape recovery is an important issue in the field of computer vision. Shape from Focus (SFF) is one of the passive techniques that uses focus information to estimate the three-dimensional shape of an object in the scene. Images are taken at multiple positions along the optical axis of the imaging device and are stored in a stack. In order to reconstruct the three dimensional shape of the object, the best-focused positions are acquired by maximizing the focus curves obtained via application of a focus measure operator. In this article, Deep Neural Network (DNN) is employed to extract the more accurate depth of each object point in the image stack. The size of each image in the stack is first reduced and then provided to the proposed DNN network to aggregate the shape. The initial shape is refined by applying a median filter, and later the reconstructed shape is sized back to original by utilizing bi-linear interpolation. The results are compared with commonly used focus measure operators by employing root mean squared error (RMSE), correlation, and image quality index (Q). Compared to other methods, the proposed SFF method using DNN shows higher precision and low computational time consumption.

Focus Measurement in Color Space for Shape From Focus Systems

“…FMs are divided into six significant groups depending upon their operating principles. Gradient-based operators take the first derivative of an image to compute the focus value, [13]- [16]. Laplacian-based operators measure the focus by calculating the second derivative of an image, [15]- [19].…”

“…Laplacian-based operators measure the focus by calculating the second derivative of an image, [15]- [19]. Wavelet-based operators consider the wavelet transform to describe an image's spatial and frequency contents, [15], [20], [21], whereas, statisticalbased operators take into account different statistics of an image to measure the focus, [22], [23], [19], [16]. Discrete cosine transform (DCTE) based operators consider DCT coefficients from the frequency content of the image to calculate the focus level, [24], [25], and finally, many other operators, which do not lie in any of the aforementioned categories, are grouped into a category named as miscellaneous operators, [26], [27].…”

“…Laplacian operators can be applied to detect the regions of high contrast. They have been used as a focus measure in both Auto-focus, [28]- [30], and SFF, [15], [16], [18], [19], [31]. They are used to compute the second derivative for each pixel of the given image window.…”

“…where Γ(i, j) is the gray value of the pixel (p (i,j) ) at coordinates (i, j), and N (io,jo) is local neighborhood around p (i,j) in the window of ω × ω. Modified versions of Laplacian operators are further studied in, [16], [32], [31].…”

Depth Hypotheses Fusion through 3D Weighted Least Squares in Shape from Focus

“…Recently, Ali et al (2019) and Ali & Mahmood (2020) have proposed to improve the focus volumes through regularization. Mutahira et al (2020) employed a simple deep neural network (DNN) to aggregate 3D shape. Ma et al (2020) tackled the SFF problem in two steps: first, depth reconstruction as a maximum a posteriori (MAP) estimation, and second, depth refinement using the Markov Random Field (MRF).…”