MeshFormer: High‐resolution Mesh Segmentation with Graph Transformer

Abstract

Graph transformer has achieved remarkable success in graph‐based segmentation tasks. Inspired by this success, we propose a novel method named MeshFormer for applying the graph transformer to the semantic segmentation of high‐resolution meshes. The main challenges are the large data size, the massive model size, and the insufficient extraction of high‐resolution semantic meanings. The large data or model size necessitates unacceptably extensive computational resources, and the insufficient semantic meanings lead to inaccurate segmentation results. MeshFormer addresses these three challenges with three components. First, a boundary‐preserving simplification is introduced to reduce the data size while maintaining the critical high‐resolution information in segmentation boundaries. Second, a Ricci flow‐based clustering algorithm is presented for constructing hierarchical structures of meshes, replacing many convolutions layers for global support with only a few convolutions in hierarchy structures. In this way, the model size can be reduced to an acceptable range. Third, we design a graph transformer with cross‐resolution convolutions, which extracts richer high‐resolution semantic meanings and improves segmentation results over previous methods. Experiments show that MeshFormer achieves gains from 1.0% to 5.8% on artificial and real‐world datasets.

MeT: A graph transformer for semantic segmentation of 3D meshes

“…ShapeNet Shapeboost [71] 77.2 [67] 77.6 ShapePFCN [72] 85.7 LaplacianNet [68] 91.5 MeshTransformer [57] 92.6 MeT (Ours) 94.2 percent points over the baseline, i.e., the model using only triangle information.…”

“…It introduces a graph transformer architecture with four new properties compared to the standard model, which are: 1) an attention mechanism which is a function of the neighborhood connectivity for each node in the graph; 2) positional encoding represented by the Laplacian eigenvectors, which naturally generalize the sinusoidal positional encoding often used in NLP; 3) a batch normalization layer in contrast to the layer normalization; 4) edge feature representation. MeshFormer [57] propose a mesh segmentation method based on graph transformers, which uses a boundary-preserving simplification to reduce the data size, a Ricci flow-based clustering algorithm for constructing hierarchical structures of meshes, and a graph transformer with cross-resolution convolutions, which extracts richer high-resolution semantic. Recently [58] introduced a novel method for 3D mesh segmentaion named Navigation Geodesic Distance Transformer (NGD-Transformer).…”