Entropic metric alignment for correspondence problems

Abstract

Many shape and image processing tools rely on computation of correspondences between geometric domains. Efficient methods that stably extract "soft" matches in the presence of diverse geometric structures have proven to be valuable for shape retrieval and transfer of labels or semantic information. With these applications in mind, we present an algorithm for probabilistic correspondence that optimizes an entropy-regularized Gromov-Wasserstein (GW) objective. Built upon recent developments in numerical optimal transportation, our algorithm is compact, provably convergent, and applicable to any geometric domain expressible as a metric measure matrix. We provide comprehensive experiments illustrating the convergence and applicability of our algorithm to a variety of graphics tasks. Furthermore, we expand entropic GW correspondence to a framework for other matching problems, incorporating partial distance matrices, user guidance, shape exploration, symmetry detection, and joint analysis of more than two domains. These applications expand the scope of entropic GW correspondence to major shape analysis problems and are stable to distortion and noise.

GWCNN: A Metric Alignment Layer for Deep Shape Analysis

“…This is problematic for shape parameterization as a customizable unit in a network, since at some level the output is a permutation. To overcome this issue, we use the metric alignment method of Solomon et al [SPKS16] because its use of entropic regularization makes the objective differentiable in the input metric spaces. This differentiability is also leveraged by Peyré et al [PCS16] to compute barycenters of sets of metric spaces.…”

“…First, since many network architectures are available for images, we define the canonical domain Σ 0 to be n 0 points laid out on the 2D grid, and encode the k output features as a multi‐channel image f 0 : Σ 0 → ℝ k on this grid. Second, to allow for diverse input representations, we use the Gromov–Wasserstein (GW) generalized mapping algorithm [SPKS16]. The GW algorithm represents a map as a “soft correspondence,” namely given two geometries Σ,Σ 0 with n and n 0 points respectively, the algorithm constructs a matrix such that a pair of points ( p,p 0 ) ∊ Σ × Σ 0 is assigned a high probability if they should be matched.…”

“…Our main contribution is a parametric and differentiable mapping layer that can be optimized for a specific problem and dataset. Our key idea is to leverage an efficient algorithm that optimizes the regularized Gromov–Wasserstein (GW) objective [SPKS16] to map from unstructured data to a regular representation. Unlike most correspondence algorithms, this regularized technique is differentiable in the geometries of the mapped domains, making it amenable to gradient‐based optimization techniques.…”

“…This yields a multi‐channel image over the grid that we feed into standard deep architectures. As we optimize the GW objective [SPKS16], our input is given in the form of a pairwise distance matrix and thus our layer can handle polygonal meshes, point clouds or general graphs as long as pairwise distances are computable. Further, the GW map minimizes distortion in pairwise distances between the source and the target, leading to mapped features that are consistent under isometric deformation of the source geometry.…”

Region-Based Correspondence Between 3D Shapes via Spatially Smooth Biclustering

“…This includes methods for computing mappings between probability densities [23,38,39], often using formal and computational tools from optimal transport theory and measure coupling. The correspondences between probability density functions produced by these methods can, in some cases, be further refined to obtain point-wise maps [37,40].…”