Dynamic Graph CNN for Learning on Point Clouds

Wang, Yue; Sun, Yongbin; Liu, Ziwei; Sarma, Sanjay E.; Bronstein, Michael M.; Solomon, Justin

doi:10.48550/arxiv.1801.07829

Cited by 240 publications

(507 citation statements)

References 55 publications

Supporting

Mentioning

505

Contrasting

Unclassified

Order By: Relevance

“…Backbone architectures. We incorporate the FA framework with two existing popular point cloud network layers: i) PointNet (Qi et al, 2017a); and ii) DGCNN (Wang et al, 2018). We denote both architectures by…”

Section: Point Clouds Euclidean Motionsmentioning

confidence: 99%

“…We instantiate the FA framework by considering different choices of symmetry groups G, their actions on data spaces V, W (manifested by choices of group representations), and the backbone architectures (or part thereof) φ, Φ we want to make invariant/equivariant to G. We consider: (i) Multi-Layer Perceptrons (MLP), and Graph Neural Networks (GNNs) with node identification (Murphy et al, 2019;Loukas, 2020) adapted to permutation invariant Graph Neural Networks (GNNs); (ii) Message-Passing GNN (Gilmer et al, 2017) adapted to be invariant/equivariant to Euclidean motions, E(d); (iii) Set network, DeepSets and PointNet (Zaheer et al, 2017;Qi et al, 2017a) adapted to be equivariant or locally equivariant to E(d); (iv) Point cloud network, DGCNN (Wang et al, 2018), adapted to be equivariant to E(d).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Frame Averaging for Invariant and Equivariant Network Design

Puny¹,

Atzmon²,

Ben-Hamu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Many machine learning tasks involve learning functions that are known to be invariant or equivariant to certain symmetries of the input data. However, it is often challenging to design neural network architectures that respect these symmetries while being expressive and computationally efficient. For example, Euclidean motion invariant/equivariant graph or point cloud neural networks. We introduce Frame Averaging (FA), a general purpose and systematic framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. Our framework builds on the well known group averaging operator that guarantees invariance or equivariance but is intractable. In contrast, we observe that for many important classes of symmetries, this operator can be replaced with an averaging operator over a small subset of the group elements, called a frame. We show that averaging over a frame guarantees exact invariance or equivariance while often being much simpler to compute than averaging over the entire group. Furthermore, we prove that FA-based models have maximal expressive power in a broad setting and in general preserve the expressive power of their backbone architectures. Using frame averaging, we propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs. We demonstrate the practical effectiveness of FA on several applications including point cloud normal estimation, beyond 2-WL graph separation, and n-body dynamics prediction, achieving state-of-the-art results in all of these benchmarks.

show abstract

Section: Point Clouds Euclidean Motionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Frame Averaging for Invariant and Equivariant Network Design

Puny¹,

Atzmon²,

Ben-Hamu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…TextureNet [36] parameterizes the room surface into local planar patches in the 4-RoSy field such that standard CNNs [7] can be applied to extract high-resolution texture information from mesh facets. Schult et al [15] applied the spatial graph convolutions of dynamic filters [31], [65], [68], [79] to the union of neighborhoods in both geodesic and Euclidean domains for vertex-wise feature learning. VMNet [80] combines the SparseConvNet [56] with graph convolutional networks to learn merged features from point clouds and meshes.…”

Section: Convolution On 3d Meshesmentioning

confidence: 99%

Geometric Feature Learning for 3D Meshes

Lei¹,

Akhtar²,

Shah³

et al. 2021

Preprint

View full text Add to dashboard Cite

Geometric feature learning for 3D meshes is central to computer graphics and highly important for numerous vision applications. However, deep learning currently lags in hierarchical modeling of heterogeneous 3D meshes due to the lack of required operations and/or their efficient implementations. In this paper, we propose a series of modular operations for effective geometric deep learning over heterogeneous 3D meshes. These operations include mesh convolutions, (un)pooling and efficient mesh decimation. We provide open source implementation of these operations, collectively termed Picasso. The mesh decimation module of Picasso is GPU-accelerated, which can process a batch of meshes on-the-fly for deep learning. Our (un)pooling operations compute features for newly-created neurons across network layers of varying resolution. Our mesh convolutions include facet2vertex, vertex2facet, and facet2facet convolutions that exploit vMF mixture and Barycentric interpolation to incorporate fuzzy modelling. Leveraging the modular operations of Picasso, we contribute a novel hierarchical neural network, PicassoNet-II, to learn highly discriminative features from 3D meshes. PicassoNet-II accepts primitive geometrics and fine textures of mesh facets as input features, while processing full scene meshes. Our network achieves highly competitive performance for shape analysis and scene parsing on a variety of benchmarks. We release Picasso and PicassoNet-II on Github.

show abstract

“…[49], 3D deep learning on point clouds has stimulated the interest of researchers. Existing methods can be mainly divided into: point-based [51,40,63,9], volumetric-based [50,44], graph-based [70,36,35,66,8,27], and viewbased [61,62,77] methods. However, volumetric-based and view-based methods lose fine-grained geometric information due to voxelization and projection, while graph-based methods are not suitable for sparse point clouds since few points cannot provide sufficient local geometric information for constructing a graph.…”

Section: Related Workmentioning

confidence: 99%

“…On the one hand, we exploit the max pooling combined with fully connected layers to capture global shape information. On the other hand, we adopt EdgeConv [70] to capture local geometric information of the target. After that, we augment the local feature of each point with the global shape information, yielding a new feature map of size 2048 × 2C.…”

Section: Shape-aware Feature Learningmentioning

confidence: 99%

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Hui¹,

Wang²,

Cheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

3D object tracking in point clouds is still a challenging problem due to the sparsity of LiDAR points in dynamic environments. In this work, we propose a Siamese voxel-to-BEV tracker, which can significantly improve the tracking performance in sparse 3D point clouds. Specifically, it consists of a Siamese shape-aware feature learning network and a voxel-to-BEV target localization network. The Siamese shape-aware feature learning network can capture 3D shape information of the object to learn the discriminative features of the object so that the potential target from the background in sparse point clouds can be identified. To this end, we first perform template feature embedding to embed the template's feature into the potential target and then generate a dense 3D shape to characterize the shape information of the potential target. For localizing the tracked target, the voxel-to-BEV target localization network regresses the target's 2D center and the z-axis center from the dense bird's eye view (BEV) feature map in an anchor-free manner. Concretely, we compress the voxelized point cloud along z-axis through max pooling to obtain a dense BEV feature map, where the regression of the 2D center and the z-axis center can be performed more effectively. Extensive evaluation on the KITTI and nuScenes datasets shows that our method significantly outperforms the current state-of-the-art methods by a large margin. Code is available at https: //github.com/fpthink/V2B.

show abstract

Dynamic Graph CNN for Learning on Point Clouds

Cited by 240 publications

References 55 publications

Frame Averaging for Invariant and Equivariant Network Design

Frame Averaging for Invariant and Equivariant Network Design

Geometric Feature Learning for 3D Meshes

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Contact Info

Product

Resources

About