Direct computation of shape cues using scale-adapted spatial derivative operators

Abstract

This paper addresses the problem of computing cues to the three-dimensional structure of surfaces in the world directly from the local structure of the brightness pattern of either a single monocular image or a binocular image pair.It is shown that starting from Gaussian derivatives of order up to two at a range of scales in scale-space, local estimates of (i) surface orientation from monocular texture foreshortening, (ii) surface orientation from monocular texture gradients, and (iii) surface orientation from the binocular disparity gradient can be computed without iteration or search, and by using essentially the same basic mechanism.The methodology is based on a multi-scale descriptor of image structure called the windowed second moment matrix, which is computed with adaptive selection of both scale levels and spatial positions. Notably, this descriptor comprises two scale parameters a local scale parameter describing the amount of smoothing used in derivative computations, and an integration scale parameter determining over how large a region in space the statistics of regional descriptors is accumulated.Experimental results for both synthetic and natural images are presented, and the relation with models of biological vision is brie y discussed.

Untitled

“…Earlier presentations of different parts of this material have appeared elsewhere (Lindeberg 1993b(Lindeberg , 1994a(Lindeberg , 1994d(Lindeberg , 1996b as well as applications of the general ideas to various problem domains Gårding 1993, 1997;Gårding and Lindeberg 1996;Li 1995, 1997;Lindeberg 1998, 1997;Almansa and Lindeberg 1996;Wiltschi et al 1997;Lindeberg 1997). The subject of this paper is to present a coherent description of the proposed scale selection methodology in journal form, including the developments and refinements that have been performed since the earliest presented manuscripts.…”

“…In (Lindeberg and Gårding 1993;Gårding and Lindeberg 1996) an extension of this general blob detection idea is presented, where: (i) each scale-space maximum is used for guiding the computation of a regional image texture descriptor (a second moment matrix) as a pre-processing stage to shape-from-texture, (ii) the shape of each blob is represented by an ellipse with its shape determined from the local statistics of image gradient directions, and (iii) the scale information is used as a cue to threedimensional surface shape when it can be assumed that the texture elements on the surface have the same size.…”

Image Matching Using Generalized Scale-Space Interest Points

“…In Lindeberg and Gårding [114], Gårding and Lindeberg [47] scale invariant blob detection by scale-space extrema was combined with subsequent computation of scale-adaptive second moment matrices to provide image features for deriving cues to local surface shape by shapefrom-texture and shape-from-disparity gradients. In Lindeberg and Gårding [115] the notion of affine shape adaptation was proposed and was demonstrated to improve the accuracy of local surface orientation estimates by computing them at affine invariant fixed points in affine scale space (Lindeberg [95, chapter 15]).…”

“…Förstner and Gülch [45] have defined other closely related measures of feature strength from the second-moment matrix. Bigün [12] has used a complex-valued generalized structure tensor for detecting different types of local symmetries in image data; see also Bigün and Granlund [13], Jähne et al [58], Lindeberg [95], Granlund and Knutsson [51], Gårding and Lindeberg [47] and Weickert [154] for other applications of the second moment matrix/structure tensor for computing local features from image data.…”

3D Object Modeling and Recognition from Photographs and Image Sequences

“…In turn, detection must be followed by a description stage that constructs a region representation invariant under these changes. For small patches of smooth Lambertian surfaces, the transformations are (to first order) affine, and we use the approach recently proposed by Mikolajczyk and Schmid to find the corresponding affine regions: Briefly, the algorithm iterates over steps where (1) an elliptical image region is deformed to maximize the isotropy of the corresponding brightness pattern (shape adaptation [10]); (2) its characteristic scale is determined as a local extremum of the normalized Laplacian in scale space (scale selection [17]); and (3) the Harris operator [12] is used to refine the position of the the ellipse's center (localization [24]). The scale-invariant interest point detector proposed in [23] provides an initial guess for this procedure, and the elliptical region obtained at convergence can be shown to be covariant under affine transformations.…”