Non-rigid Shape Matching Using Geometry and Photometry

Abstract

Abstract. In this paper, we tackle the problem of finding correspondences between three-dimensional reconstructions of a deformable surface at different time steps. We suppose that (i) the mechanical underlying model imposes time-constant geodesic distances between points on the surface; and that (ii) images of the real surface are available. This is for instance the case in spatio-temporal shape from videos (e.g. multiview stereo, visual hulls, etc.) when the surface is supposed approximatively unstretchable. These assumptions allow to exploit both geometry and photometry. In particular we propose an energy based formulation of the problem, extending the work of Bronstein et al.[1]. On the one hand, we show that photometry (i) improves accuracy in case of locally elastic deformations or noisy surfaces and (ii) allows to still find the right solution when [1] fails because of ambiguities (e.g. symmetries). On the other hand, using geometry makes it possible to match shapes that have undergone large motion, which is not possible with usual photometric methods. Numerical experiments prove the efficiency of our method on synthetic and real data.

Discrete Minimum Distortion Correspondence Problems for Non-rigid Shape Matching

“…Using structures invariant to pre-defined classes of transformations (or, using their statistical distributions if such transformations cannot be modeled explicitly) allows obtaining invariant matching between shapes. Our approach generalizes many previous works in the field, in particular, methods based on metric distortion minimization [2,3,17] and global and local features [9,13,20], allowing incorporating many existing geometries and local descriptors [5,7,8]. In particular, it extends the GromovHausdorff framework [2,3,19].…”

“…Examples of such structures include multiscale heat kernel signatures [5][6][7], local photometric properties [8,9], eigenfunctions of the Laplace-Beltrami operator [10][11][12][13], triplets of points [14,15], and geodesic [2,3,16], diffusion [17], and commute time [10,18] distances. By defining a structure invariant under certain class of transformations (e.g.…”

“…He introduced the Gromov-Wasserstein distance based on measure coupling between two metric-measure spaces, and formulated it as a quadratic assignment problem (QAP). Thorstensen and Keriven [9] extended the GMDS framework to textured shapes introducing photometric stress as a local matching term in addition to geodesic distance distortion. Dubrovina and Kimmel [13] generalized this approach for the matching of textureless shapes using LaplaceBeltrami eigenfunctions as local geometric descriptors.…”

Hierarchical Matching of Non-rigid Shapes

“…Bronstein et al . [7] used the approach suggested in [6] with diffusion geometry, in order to match shapes with topological noise, and Thorstensen and Keriven [35] extended it to handle surfaces with textures. The methods in [24,22,23] were intended for surface comparison rather than matching, and as such they do not produce correspondence between shapes.…”

A game-theoretic approach to deformable shape matching

“…A large corpus of research makes use of the notion of intrinsic geometry -an umbrella term referring to geometric structures that remain invariant under non-rigid bendings and other types of transformations. In [7,12,4,11,18] and followup studies, it was proposed to use the distortion of intrinsic metrics as a measure of the correspondence quality. Finding a minimum distortion correspondence can be rigorously formulated in geometric terms and cast as an optimization problem.…”