Image Matching Using Generalized Scale-Space Interest Points

Abstract

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian ∇ 2 norm L, the determinant of the Hessian det H norm L and four new unsigned or signed Hessian feature strength measures D 1,norm L,D 1,norm L, D 2,norm L and D 2,norm L, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian det H norm L is a truly affine covariant differential entity and the Hessian feature strength measures D 1,norm L andD 1,norm L have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differen- It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures D 1,norm L,D 1,norm L and the determinant of the Hessian det H norm L being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris-Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.

Spatio-Temporal Scale Selection in Video Data

“…Given the spatial Gaussian scale-space concept [24,34,44,46,47,59,60,67,70,106,111,120,123], a general methodology for spatial scale selection has been developed based on local extrema over spatial scales of scale-normalized differential entities [62,64,65,72,73]. This general method- 2 The spatial Laplacian applied to the first-and second-order temporal derivatives ∇ 2 (x,y) L t and ∇ 2 (x,y) L tt as well as the spatio-temporal Laplacian ∇ 2 (x,y,t) L computed from a video sequence in the UCF-101 dataset (Kayaking_g01_c01.avi) at 3 × 3 combinations of the spatial scales (bottom row) σ s,1 = 2 pixels, (middle row) σ s,2 = 4.6 pixels and (top row) σ s,3 = 10.6 pixels and the temporal scales (left column) σ τ,1 = 40 ms, (middle column) σ τ,2 = 160 ms and (right column) σ τ,3 = 640 ms with the spatial and temporal scale parameters in units of σ s = √ s and σ τ = √ τ and using a time-causal spatio-temporal scale-space representation with a logarithmic distribution of the temporal scale levels for c = 2 (image size: 320 × 172 pixels of original 320 × 240 pixels; frame 90 of 226 frames at 25 framesframes/s) ology has in turn been successfully applied to develop robust methods for image-based matching and recognition [5,41,52,68,74,84,86,87,89,90,[112][113][114] that are able to handle large variations of the size of the objects in the image domain and with numerous applications regarding object recognition, object categorization, multi-view geometry, construction of 3-D models from visual input,…”

“…To begin, we will start developing our theory for spatiotemporal scale selection with respect to the problem of detecting sparse spatio-temporal interest points [6,9,11,14,18,20,21,30,49,88,94,97,99,100,107,122,124,126,127], which may be regarded as a conceptually simplest problem domain because of the sparsity of spatio-temporal interest points and the close connection between this problem domain and the detection of spatial interest points for which there exists a theoretically well-founded and empirically tested framework regarding scale selection over the spatial domain [1,4,5,15,17,25,42,65,72,74,84,89,90,112]. Specifically, using a non-causal Gaussian spatio-temporal scale-space model, we will perform a theoretical analysis of the spatio-temporal scale selection properties of eight different types of spatiotemporal interest point detectors and show that seven of them: (i) the spatial Laplacian of the first-order temporal derivative, (ii) the spatial Laplacian of the second-order temporal derivative, (iii) the determinant of the spatial Hessian of the first-order temporal derivative, (iv) the determinant of the spatial Hessian of the second-order temporal derivative, (v) the determinant of the spatio-temporal Hessian matrix, (vi) the first-order temporal derivative of the determinant of the spatial Hessian matrix and (vii) the second-order temporal derivative of the determinant of the spatial Hessian matrix, do all lead to fully scale-covariant spatio-temporal scale estimates and scale-invariant feature responses under independent scaling transformations of the spatial and the temporal domains.…”

“…Inspired by the way the determinant of the spatial Hessian matrix constitutes a better spatial interest point detector than the spatial Laplacian operator [74], we consider an extension of the spatial Laplacian of the second-order temporal derivative (42) into the determinant of the spatial Hessian applied to the second-order temporal derivative…”

Image Matching Using Generalized Scale-Space Interest Points

“…Acknowledgments An earlier version of this work was presented at the SSVM 2013 conference [108]. I would like to thank Lars Bretzner for his help when preparing the poster image dataset and Oskar Linde for sharing his code for local image descriptors.…”

Casamento de modelos baseado em projeções radiais e circulares invariante a pontos de vista.

“…Como resultado, as análises realizadas numa única escala podem perder informação. Uma solução é analisar em todas as escalas (ADELSON, et al, 1984;LINDEBERG, 2015).…”