Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling

Abstract

Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, Point-Net has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with a more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy to a convolution kernel for images, we define a point-set kernel as a set of learnable 3D points that jointly respond to a set of neighboring data points according to their geometric affinities measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local high-dimensional feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network can efficiently capture local information and robustly achieve better performances on major datasets. Our code is available at http://www.merl.com/research/ license#KCNet

Point Cloud Processing via Recurrent Set Encoding

“…In particular, we can sort the k nearest neighbor points according to their distances to the query point, and then apply RNN to this point sequence for local feature learning. This is different from KCNet (Shen et al 2018) which uses a local point-set kernel, and will be explored in the future.…”

“…Representative works include PointNet (Qi et al 2017a) and PointNet++ (Qi et al 2017b), which process point clouds by combining multilayer perceptron (MLP) network with symmetric operations (e.g., max-pooling) to learn point features globally or hierarchically. Inspired by PointNet, several recent methods have been proposed to further improve the point feature representation (Shen et al 2018;Xie et al 2018;Li, Chen, and Lee 2018). This class of networks are invariant to input permutation and have achieved state-of-the-art results.…”

Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds

“…The channel settings of the first MLP and the six SPH3D layers is 32 and 64-64-64-128-128-128. We use the same classifier 512-256-40 as the previous works [32], [33], [41]. The Encoder4 in Table 2 indicates that the network learns a global representation of the point cloud using G-SPH3D.…”

SRINet

“…Since pre-multiplying original feature matrix by the learned transformation matrix is equal to swap or linearly recombine features in each row of original matrix, permutation equivalence can not be guaranteed. KCNet [19] proposes a shallow Kernel Correlation (KC) layer and K-NN Graph to incorporate local feature. We believe that deepening the KC layer may help mine more abstract and discriminative signals.…”

“…In PointNet, transform matrix is learned from the whole points to integrate the features attached to every vertex, which is considered to impair the capability of perceiving local structure [17,19]. Thus, it is necessary to extract features from local regions.…”