Evaluating Feature Extraction Methods with Synthetic Noise Patterns for Image-Based Modelling of Texture-Less Objects

Abstract

Image-based three-dimensional (3D) reconstruction is a process of extracting 3D information from an object or entire scene while using low-cost vision sensors. A structure-from-motion coupled with multi-view stereo (SFM-MVS) pipeline is a widely used technique that allows 3D reconstruction from a collection of unordered images. The SFM-MVS pipeline typically comprises different processing steps, including feature extraction and feature matching, which provide the basis for automatic 3D reconstruction. However, surfaces with poor visual texture (repetitive, monotone, etc.) challenge the feature extraction and matching stage and affect the quality of reconstruction. The projection of image patterns while using a video projector during the image acquisition process is a well-known technique that has been shown to be successful for such surfaces. In this study, we evaluate the performance of different feature extraction methods on texture-less surfaces with the application of synthetically generated noise patterns (images). Seven state-of-the-art feature extraction methods (HARRIS, Shi-Tomasi, MSER, SIFT, SURF, KAZE, and BRISK) are evaluated on problematic surfaces in two experimental phases. In the first phase, the 3D reconstruction of real and virtual planar surfaces evaluates image patterns while using all feature extraction methods, where the patterns with uniform histograms have the most suitable morphological features. The best performing pattern from Phase One is used in Phase Two experiments in order to recreate a polygonal model of a 3D printed object using all of the feature extraction methods. The KAZE algorithm achieved the lowest standard deviation and mean distance values of 0.0635 mm and −0.00921 mm, respectively.

Exploiting Light Directionality for Image‐Based 3D Reconstruction of Non‐Collaborative Surfaces

“…On the contrary, photogrammetry still performs poorly on objects with shiny or textureless surfaces. To resolve the problem, the authors proposed to project a known (Hafeez et al., 2017, 2020; Menna et al., 2017), random (Hosseininaveh et al., 2015; Ahmadabadian et al., 2017, 2019) or synthetic (Hafeez et al., 2018, 2020; Santoši et al., 2019) pattern onto the object. Menna et al.…”

“…On the contrary, photogrammetry still performs poorly on objects with shiny or textureless surfaces. To resolve the problem, the authors proposed to project a known (Hafeez et al, 2017(Hafeez et al, , 2020Menna et al, 2017), random (Hosseininaveh et al, 2015;Ahmadabadian et al, 2017Ahmadabadian et al, , 2019 or synthetic (Hafeez et al, 2018(Hafeez et al, , 2020Santo si et al, 2019) pattern onto the object. Menna et al (2016) created an automated workflow based on pattern projection for 3D digitisation of heritage artifacts: various known random patterns are used to enhance dense image-matching.…”

Investigating 3d Reconstruction of Non-Collaborative Surfaces Through Photogrammetry and Photometric Stereo

“…Over the years, different photogrammetric methods have been developed to deal with the 3D reconstruction of noncollaborative objects. For 3D reconstruction of Lambertian textureless objects, previous studies have been concentrated on improving surface texture by projecting, for example, a known pattern (Menna et al, 2017;Mousavi et al, 2018), random (Hosseininaveh et al, 2015;Ahmadabadian et al, 2019) or a synthetic texture (Santoši et al, 2019;Hafeez et al, 2020) onto the objects. These methods, however, assume that the object surface is Lambertian, which is not the case with objects that have specular reflection or interreflection effects.…”

“…However, it is still challenging to achieve high-accuracy 3D measurement of non-collaborative objects (Figure 1) due to the sensitivity of photogrammetry to the textural properties of the surface (e.g., opaque, translucent, roughness). Consequently, noisy results on poorly textured objects are typically generated (Ahmadabadian et al, 2017;Santoši et al, 2019;Hafeez et al, 2020). Unlike photogrammetry approaches, photometric stereo can recover a very detailed topography of objects even with textureless or shiny surfaces (Li et al, 2020;Wang et al, 2020;Wei et al, 2020).…”

A Practical 3D Reconstruction Method for Weak Texture Scenes

“…Random-pattern-based systems apply noise patterns to the weakly textured plane and use SFM-MVS algorithms to generate 3D models. However, the quality of reconstruction model depends on the quality of the features [43]. Some random-pattern-based systems focus on reconstructing small objects from the geometric constraints based on matching feature points in images.…”