Spectral Shape Recovery and Analysis Via Data-driven Connections

Abstract

We introduce a novel learning-based method to recover shapes from their Laplacian spectra, based on establishing and exploring connections in a learned latent space. The core of our approach consists in a cycle-consistent module that maps between a learned latent space and sequences of eigenvalues. This module provides an efficient and effective link between the shape geometry, encoded in a latent vector, and its Laplacian spectrum. Our proposed data-driven approach replaces the need for ad-hoc regularizers required by prior methods, while providing more accurate results at a fraction of the computational cost. Moreover, these latent space connections enable novel applications for both analyzing and controlling the spectral properties of deformable shapes, especially in the context of a shape collection. Our learning model and the associated analysis apply without modifications across different dimensions (2D and 3D shapes alike), representations (meshes, contours and point clouds), nature of the latent space (generated by an auto-encoder or a parametric model), as well as across different shape classes, and admits arbitrary resolution of the input spectrum without affecting complexity. The increased flexibility allows us to address notoriously difficult tasks in 3D vision and geometry processing within a unified framework, including shape generation from spectrum, latent space exploration and analysis, mesh super-resolution, shape exploration, style transfer, spectrum estimation for point clouds, segmentation transfer and non-rigid shape matching.

Localized Shape Modelling with Global Coherence: An Inverse Spectral Approach

“…The vision community has recently rediscovered interest in this problem from a practical perspective, with [8,32] showing that this inverse problem can be solved through a complex optimization. The recent work [26], and its extension [25], replace the costly optimization with a data-driven framework, where a latent encoding is connected with the Laplacian spectrum via trainable maps. At test time, the network can instantaneously recover a shape from its spectrum.…”

“…At test time, the network can instantaneously recover a shape from its spectrum. While we consider these works the closest to ours for its data-driven nature, [26,25] limit their analysis to the standard Laplacian, without investigating localized operators [27,6]. Importantly, the bandwidth used in [26,25] (30 lowest eigenvalues) does not contain enough information on the shape geometry, and this information is spread by the network in an unclear way.…”

“…While we consider these works the closest to ours for its data-driven nature, [26,25] limit their analysis to the standard Laplacian, without investigating localized operators [27,6]. Importantly, the bandwidth used in [26,25] (30 lowest eigenvalues) does not contain enough information on the shape geometry, and this information is spread by the network in an unclear way. Finally, [26,25] propose to solve the problem as a block of a complex and hard to interpret network, hiding the correlation between the input and the output.…”

“…Inverse spectral methods [8,32] have shown that the eigenvalues of differential operators may uniquely represent a 3D object given a limited context (e.g., in the object class). One of the last advances in this field [26,25] brings together for the first time the spectra with deep learning techniques. However, several points remain open and elicit our curiosity.…”

“…However, several points remain open and elicit our curiosity. First of all, it is not clear if an autoencoder architecture is needed as in [26,25]. Nevertheless, what is indeed true, is that it blurs the relationship between the spectra and the generated models.…”