Deformation-driven shape correspondence via shape recognition

Abstract

Figure 1: We develop GRASS, a Generative Recursive Autoencoder for Shape Structures, which enables structural blending between two 3D shapes. Note the discrete blending of translational symmetries (slats on the chair backs) and rotational symmetries (the swivel legs). GRASS encodes and synthesizes box structures (bottom) and part geometries (top) separately. The blending is performed on fixed-length codes learned by the unsupervised autoencoder, without any form of part correspondences, given or computed. AbstractWe introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures. Our key insight is that 3D shapes are effectively characterized by their hierarchical organization of parts, which reflects fundamental intra-shape relationships such as adjacency and symmetry. We develop a recursive neural net (RvNN) based autoencoder to map a flat, unlabeled, arbitrary part layout to a compact code. The code effectively captures hierarchical structures of man-made 3D objects of varying structural complexities despite being fixed-dimensional: an associated decoder maps a code back to a full hierarchy. The learned bidirectional mapping is further tuned using an adversarial setup to yield a generative model of plausible structures, from which novel structures can be sampled. Finally, our structure synthesis framework is augmented by a second trained module that produces fine-grained part geometry, conditioned on global and local structural context, leading to a full generative pipeline for 3D shapes. We demonstrate that without supervision, our network learns meaningful structural hierarchies adhering to perceptual grouping principles, produces compact codes which enable applications such as shape classification and partial matching, and supports shape synthesis and interpolation with significant variations in topology and geometry.

Gyral net: A new representation of cortical folding organization

“…Based on the extracted gyral nets, an automatic searching framework will be developed to identify novel cortical landmarks such as the consistent gyral joints introduced in (Li et al., 2016). In addition, the state of art graph matching and deep learning algorithms (Li et al, 2017) can be developed to align gyral nets for group-wise analysis in the future.…”

Elastic Correspondence between Triangle Meshes

“…Our approach, in contrast, accounts for the actual distortion of the correspondence and for extrinsic features by using quantities such as edge lengths and dihedral angles, which are invariant to global rotations. Two more recent deformation‐based correspondence approaches [AXZ∗15, ZYL∗17] target mostly man made shapes consisting of parts that can be represented using simple geometric primitives, and are therefore less appropriate for non‐isometric manifold models.…”

Consistent segment-wise matching with multi-layer graphs

“…Given two similar 3D meshes with pre-defined segments, 3D segment-wise matching aims to establish meaningful correspondences of segments between the two meshes. It is an important problem as it helps with higher-level and hierarchical understanding in geometry analysis Zhu et al (2017). It further impacts many downstream applications, like defining better similarity measures between 3D models Kleiman et al (2015); Shapira et al (2010); Kleiman and Ovsjanikov (2017), functionality analysis van Kaick et al (2013a), surface registration Huang et al (2008) and structure-aware analysis Mitra et al (2013).…”

“…To solve the segment-wise matching problem, they use a deformation energy as an effective constraint to produce higher-level semantic matching results. Zhu et al (2017) builds a component hierarchical graph using a binary partition technique. Their matching technique adopts a top-down approach and achieves good results.…”