CNNs on surfaces using rotation-equivariant features

Abstract

This paper is concerned with a fundamental problem in geometric deep learning that arises in the construction of convolutional neural networks on surfaces. Due to curvature, the transport of filter kernels on surfaces results in a rotational ambiguity, which prevents a uniform alignment of these kernels on the surface. We propose a network architecture for surfaces that consists of vector-valued, rotation-equivariant features. The equivariance property makes it possible to locally align features, which were computed in arbitrary coordinate systems, when aggregating features in a convolution layer. The resulting network is agnostic to the choices of coordinate systems for the tangent spaces on the surface. We implement our approach for triangle meshes. Based on circular harmonic functions, we introduce convolution filters for meshes that are rotation-equivariant at the discrete level. We evaluate the resulting networks on shape correspondence and shape classifications tasks and compare their performance to other approaches. CCS Concepts: • Computing methodologies → Neural networks; Shape analysis.

Automated morphological phenotyping using learned shape descriptors and functional maps: A novel approach to geometric morphometrics

“…Our model differs from SURFMNet in two ways: (i) it accepts geometric data derived from rigidly-aligned polygon models as input as opposed to precalculated spectral shape descriptors, and (ii) it uses a Harmonic Surface Network (HSN) as a feature extractor in place of a fully connected residual network. We choose HSN as our feature extractor due to its ability to produce rotation-invariant features from polygon model geometry, an essential property for our descriptors (see Methods and Materials for additional details) [38]. Our network's FM layer estimates the forward and backward correspondences, C 12 and C 21 , between a source shape S 1 and a target shape S 2 .These functional maps are easily converted to dense P2P correspondences, T 12 and T 21 .…”

“…Existing methods either (i) suffer from poor expressivity or (ii) are too sensitive to differences in polygon model connectivity, or (iii) they don't produce rotation-invariant features in a manner that is conducive to learning spectral descriptors. In this study, we craft spectral descriptors in a similar point-cloud-based fashion, but we use a Harmonic Surface Network (HSN) as a feature extractor [38]. HSN-based feature extractors produce highly expressive intrinsic features that are strongly locally-aligned.…”

DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds

“…Two-stream architectures Architectures with two streams and vector-valued features are also used in rotation-equivariant approaches for images [77] and surface meshes [16,76,12,57]. These networks constrain kernels to output complex-valued and rotation-or gaugeequivariant features, which are separated into orders of equivariance.…”

“…This means that a network with DeltaConv is able to learn from directional information, while being agnostic to the choice of basis vectors in tangent spaces. This is an alternative to building reference frame fields to compare results of filters along a surface [3,53,32], as well as to methods relying on equivariant filters to achieve such independence [16,76].…”

“…We go into an orthogonal direction by building intrinsic convolutions, which operate in fewer dimensions and naturally generalize to (non-)rigidly deformed shapes. Another promising solution is to map special 2D kernels to the surface using charts (local coordinate systems) [24,12,16,75,76,57,52]. By constraining kernels to a family of rotation-or gaugeequivariant functions, these networks can relate the output of convolutions at different points by a rotation or gaugetransformation.…”