General-Purpose Deep Point Cloud Feature Extractor

3D shape retrieval is an important researching field due to its wide applications in computer vision and multimedia fields. With the development of deep learning technology, great progress has been made in recent years and lots of methods have achieved promising 3D shape retrieval results. Due to the effective description of point cloud data on structural information for 3D shapes, lots of methods based on point cloud data format are proposed for better shape representation. However, most of them focus on extracting a global descrisptor from the whole 3D shape while the local features and detailed structural information are ignored, which negatively affect the effectiveness of shape descriptors. In addition, these methods also ignore the correlations among different parts of point clouds, which may introduce redundant information to the final shape descriptors. In order to address these issues, we propose a Multipart attention network (MPAN) for 3D model retrieval based on point cloud. Firstly, we segment a 3D shape into multiple parts by employing a pre-trained PointNet++ segmentation model. After extracting the local features from them, we introduce a novel self-attention mechanism to explore the correlations between different parts. Meanwhile, by considering the structural relevance of them, the redundancy for representing 3D shapes is removed while the effective information is utilized. Finally, informative and discriminative shape descriptors, considering both local features and spatial correlations, are generated for 3D shape retrieval task. To validate the effectiveness of our method, we conduct several experiments on the public 3D shape benchmark, ShapeNetPart dataset. Experimental results and comparisons with state-of-the-art methods demonstrate the superiority of our proposed method.

Section: A Model-based Methodsmentioning

confidence: 99%

MPAN: Multi-Part Attention Network for Point Cloud Based 3D Shape Retrieval

Zhao

et al. 2020

“…• Our method is consistently competitive compared with other representative view-based and model-based methods for both 3D model retrieval and classification tasks, which demonstrates the superiority and efficiency of our proposed method. [9] 77.0% -3D-GAN [27] 83.3% -VSL [28] 84.5% -Shape-based binVoxNetPlus [29] 85.47% -PointNet [30] 89.2% kd-Networks [31] 91.8% -3D-A-Nets [32] 90.5% 80.1% G3DNet [13] 91.13% -PointNet++ [33] 91.9% -DeepPano [34] 77.6% 76.8% GIFT [17] 83.1% 81.9% Geometry Image [35] 83.9% 53.1% View-based Multiple Depth Maps [36] 87.8% -MVCNN [18] 90.1% 79.5% PANORAMA-NN [37] 90.7% 83.5% Pariwise [38] 90.7% -MVCNN-MultiRes [39] 91.4% -MVTS (Our) 93.4% 87.3% • Previous view-based methods usually just select one representative view from the view sequence of the model, or employ simple view-level aggregation strategy, like the max-pooling (eg. MVCNN) method to fuse multiple views.…”

Section: A Comparison With the State-of-the-art Methodsmentioning

confidence: 99%

“…Qi et al [12] proposed to the make use of point cloud descriptions for 3D models for classification. Dominguez et al [13] proposed the graph-based method to apply transfer learning strategy on 3D point cloud data, which was demonstrated to have the ability to represent the unforeseen test models. You et al [14] integrated point clouds and multi-view data into 3D model recognition.…”

Section: A Model-based 3d Model Retrievalmentioning

confidence: 99%

Multi-View Tree Structure Learning for 3D Model Retrieval and Classification in Smart City

Liu

Zhao

et al. 2020

The application of digital products in smart city results in ever-increasing 3D model data and how to obtain the relevant 3D model becomes a crucial issue. In this paper, we propose the Multi-View Tree Structure (MVTS) learning for 3D model retrieval and recognition. MVTS contains three key consecutive modules. Firstly, the visual feature learning module extracts the visual features of multiple views. Then, we design a score matrix to estimate the value of contextual information between view pairs. Based on the score matrix, a maximum spanning tree is constructed to further explore the contextual information within multiple views. Then, we utilize the bidirectional Tree-LSTM to encode the contextual information among views and the spatial information of tree structure and optimize the tree parameters. After that, the tree attention strategy is adopted to explore the importance of each view. Comparing to existing methods, our proposed method explores the spatial information of 3D model without the requirement of specific camera settings, which is more suitable for real applications. Moreover, our method jointly realizes the feature learning, view-wise contextual information and tree spatial information encoding and view importance estimating, which enhances the discrimination of the 3D model representation. Extensive experimental results on Modelnet40 and ShapeNetCore55 demonstrate the superiority of our method.

“…It combines the discrete structure of the grid with the continuous generalization of Fisher vectors and has high computational efficiency. Domínguez et al [18] proposed a 3D point cloud processing method based on graphics. They attempted to use large scale transfer learning on 3D point cloud data and demonstrate the ability to identify potential performance of 3D point clouds on unpredictable test sets.…”

Section: A Model-based Methodsmentioning

confidence: 99%

Multi-View Hierarchical Fusion Network for 3D Object Retrieval and Classification

Liu

Song

et al. 2019

The rapid development of 3D technique has led to the dramatic increase in 3D data. The scalable and effective 3D object retrieval and classification algorithms become mandatory for large-scale 3D object management. One critical problem of view-based 3D object retrieval and classification is how to exploit the relevance and discrimination among multiple views. In this paper, we propose a multi-view hierarchical fusion network (MVHFN) for these two tasks. This method mainly contains two key modules. First, the module of visual feature learning applies the 2D CNNs to extract the visual feature of multiple views rendered around the specific 3D object. Then, the multi-view hierarchical fusion module we proposed is employed to fuse the multiple view features into a compact descriptor. This module can not only fully exploit the relevance among multiple views by intra-cluster multi-view fusion mechanism, but also discover the content discrimination by inter-cluster multi-view fusion mechanism. Experimental results on two public datasets, i.e., ModelNet40 and ShapeNetCore55, show that our proposed MVHFN outperforms the current state-of-the-art methods in both the 3D object retrieval and classification tasks. INDEX TERMS 3D object retrieval, 3D object classification, 3D shape recognition, multi-view.