Sampling Based on Kalman Filter for Shape from Focus in the Presence of Noise

Abstract

Recovering three-dimensional (3D) shape of an object from two-dimensional (2D) information is one of the major domains of computer vision applications. Shape from Focus (SFF) is a passive optical technique that reconstructs 3D shape of an object using 2D images with different focus settings. When a 2D image sequence is obtained with constant step size in SFF, mechanical vibrations, referred as jitter noise, occur in each step. Since the jitter noise changes the focus values of 2D images, it causes erroneous recovery of 3D shape. In this paper, a new filtering method for estimating optimal image positions is proposed. First, jitter noise is modeled as Gaussian or speckle function, secondly, the focus curves acquired by one of the focus measure operators are modeled as a quadratic function for application of the filter. Finally, Kalman filter as the proposed method is designed and applied for removing jitter noise. The proposed method is experimented by using image sequences of synthetic and real objects. The performance is evaluated through various metrics to show the effectiveness of the proposed method in terms of reconstruction accuracy and computational complexity. Root Mean Square Error (RMSE), correlation, Peak Signal-to-Noise Ratio (PSNR), and computational time of the proposed method are improved on average by about 48%, 11%, 15%, and 5691%, respectively, compared with conventional filtering methods.

A simplified approach using deep neural network for fast and accurate shape from focus

“…Occasionally, the recovered 3D shapes have some bad regions caused by noise, this noise can be Image noise (Malik and Choi (2008)), or Jitter noise (Jang et al (2019)) of the SFF systems. Several techniques have been proposed in the literature to reject the regions that are affected by these types of noise (Malik and Choi (2008); Jang et al (2019); Jeon et al (2019); Muhammad & Choi, 2009). The rejected points can be recovered by applying a variety of median filters, or other surface interpolation techniques (Muhammad and Choi (2009)).…”

“…Recently, a new FM based on the analysis of 3D structure tensor of the image sequence is proposed in (Mahmood and Lee (2019)). Furthermore, the removal of jitter from sampled images in SFF using Kalman filter has also been proposed in (Jang et al [2018(Jang et al [ , 2019). Ma, Kim, and Shin 2020…”

“…Another SFF method based on combining a 3D steerable filter on treating a texture‐less region is proposed in (Fan and Yu (2018)). Since the image sequence may contain jitter noise caused by mechanical vibrations, a Kalman filter is designed to eliminate this noise and hence, improve the quality of the recovered shapes in (Jang, Muhammad, & Choi, 2018; Jang, Muhammad, Yun, & Kim, 2019). The quality of the reconstructed 3D shapes by SFF, often depends on the applied FM.…”

“…Recently, a new FM based on the analysis of 3D structure tensor of the image sequence is proposed in (Mahmood and Lee (2019)). Furthermore, the removal of jitter from sampled images in SFF using Kalman filter has also been proposed in (Jang et al [2018, 2019, 2020, b]). Ma, Kim, and Shin (2020) proposed a method for depth reconstruction using nonlocal matting Laplacian prior with Markov random field to obtain a reliable depth image with clear edges and fine details.…”

Removal of Non-Gaussian Jitter Noise for Shape From Focus Through Improved Maximum Correntropy Criterion Kalman Filter

“…This jitter noise is different from image noise, and it changes focus values along the optical axis due to random vibrations, thus, degrading the performance of 3D shape recovery. Filtering methods, such as Kalman filter, [18], Bayes filter, [19], particle filter, [20], modified Kalman filter, [21], and adaptive neural network filter, [22], have been proposed for removing this type of jitter noise. Since the conventional filters use minimum mean squared error (MMSE) criterion as a cost function, they only capture the second-order statistics of the error signal.…”