Linear Multiscale Analysis of Similarities between Images on Riemannian Manifolds: Practical Formula and Affine Covariant Metrics

Abstract

In this paper we study the problem of comparing two patches of images defined on Riemannian manifolds which in turn can be defined by each image domain with a suitable metric depending on the image. For that we single out one particular instance of a set of models defining image similarities that was earlier studied in [C. Ballester et al., Multiscale Model. Simul., 12 (2014), pp. 616-649], using an axiomatic approach that extended the classicalÁlvarez-Guichard-Lions-Morel work to the nonlocal case. Namely, we study a linear model to compare patches defined on two images in R N endowed with some metric. Besides its genericity, this linear model is selected by its computational feasibility since it can be approximated leading to an algorithm that has the complexity of the usual patch comparison using a weighted Euclidean distance. Moreover, we propose and study some intrinsic metrics which we define in terms of affine covariant structure tensors and we discuss their properties. These tensors are defined for any point in the image and are intrinsically endowed with affine covariant neighborhoods. We also discuss the effect of discretization over the affine covariance properties of the tensors. We illustrate our theoretical results with numerical experiments. Introduction.Image comparison is a topic that has received a lot of attention in the image processing and computer vision communities since it is a main ingredient in many applications, such as object recognition, stereo vision, image interpolation, image denoising, and exemplar-based image inpainting, among others. A common way to define a nonlocal similarity measure between two images is to compare the patches (local neighborhoods) around each pair of points formed by taking one point from each image. We consider a general setting in which images are defined on Riemannian manifolds. Such manifolds arise, for instance, for images defined on R N , endowed with a suitable metric depending on the image.In [3] it was shown that multiscale analyses of similarities between images on Riemannian manifolds, satisfying a certain set of axioms, are (viscosity) solutions of a family of degenerate PDEs. Our goal in this paper is to study one particular instance of the set of models derived in [3], namely a linear model to compare patches defined on two images in R N endowed

Affine Invariant Self-similarity for Exemplar-based Inpainting

“…We propose to use an affine invariant similarity measure which automatically distorts the patches being compared (Fedorov et al, 2015). Our method considers a rich patch space that includes all affine-transformed patches, however, for each pair of patches the transformations are uniquely determined using the image content.…”

“…The high dimensionality of the parameter space makes the problem very difficult. In this paper we use an affine invariant similarity measure, introduced in (Fedorov et al, 2015), that automatically deduces this transformation from the local texture context.…”

“…We follow the approach recently proposed in (Fedorov et al, 2015), where affine covariant tensor fields computed a priori in each image are used to define an affine invariant similarity measure between patches. We incorporate this measure into a variational inpainting formulation.…”

“…Due to the affine covariance property of the tensors, these patches transform appropriately when computed on an affinely transformed version of the image. Figure 2 illustrates the patches defined by the affine covariant tensors of (Fedorov et al, 2015), computed for a set of corresponding points in two images related by a homography. Note that even though the transformation is not an affinity, the patches still match, since a homography can be locally approximated by an affinity.…”

“…Section 2 reviews the related work. In Section 3, we summarize the results of (Fedorov et al, 2015) which motivates the definition of the similarity measure we use. Section 4 is devoted to the inpainting method and algorithm we propose.…”