Masked Discrimination for Self-Supervised Learning on Point Clouds

Liu, Haotian; Cai, Mu; Lee, Yong Jae

doi:10.48550/arxiv.2203.11183

Cited by 4 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is particularly important for 3D point cloud analysis, since the collection and annotation of point cloud data are much more expensive than 2D images. Popular SSL methods for point cloud include reconstruction [68,16,75,60,8,25,76,70,34,40,73,65,15], instance contrastive feature learning [49,51], consistency feature learning against augmentations [31], and other pretext tasks [52,43,1]. Among these methods, the masked auto-encoding [60,76,70,40] has been receiving more and more attention recently.…”

Section: Related Workmentioning

confidence: 99%

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Zhang¹,

Shi²,

Wu³

2022

SSRN Journal

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Zhang¹,

Shi²,

Wu³

2022

SSRN Journal

View full text Add to dashboard Cite

“…This speeds up training compared to Point-BERT and also improves downstream performance. MaskPoint [22] further speeds up pre-training by removing the point cloud reconstruction. Instead, the decoder is trained to discriminate between masked point patches and fake, empty ones, sampled at random.…”

Section: Related Workmentioning

confidence: 99%

“…Besides the encoded and masked voxels, we also add a set of empty masked voxels, similar to what is done in [22].…”

Section: Decodermentioning

confidence: 99%

“…Recently, multiple works have applied masked point modeling techniques to pre-train point cloud encoders [16,22,27,37,39]. These have achieved favourable results on downstream tasks like shape classification, shape segmentation, few-shot classification, and indoor 3D OD, indicating the effectiveness of masked autoencoders in the point cloud domain.…”

Section: Introductionmentioning

confidence: 99%

“…Naturally, design choices like point cloud representation and model selection have been tailored toward dataset characteristics. For instance, the low number of points per scene lessens requirements on computational efficiency and has enabled the use of vanilla Transformers [22,27,37]. Further, existing methods have relied exclusively on furthest point sampling (FPS) and k-nearest neighbors (kNN) for representing point clouds and dividing them into subsets of equally many points, see Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Clouds

Heß¹,

Jaxing²,

Svensson³

et al. 2022

Preprint

View full text Add to dashboard Cite

Masked autoencoding has become a successful pretraining paradigm for Transformer models for text, images, and recently, point clouds. Raw automotive datasets are a suitable candidate for self-supervised pre-training as they generally are cheap to collect compared to annotations for tasks like 3D object detection (OD). However, development of masked autoencoders for point clouds has focused solely on synthetic and indoor data. Consequently, existing methods have tailored their representations and models toward point clouds which are small, dense and have homogeneous point density. In this work, we study masked autoencoding for point clouds in an automotive setting, which are sparse and for which the point density can vary drastically among objects in the same scene. To this end, we propose Voxel-MAE, a simple masked autoencoding pre-training scheme designed for voxel representations. We pre-train the backbone of a Transformer-based 3D object detector to reconstruct masked voxels and to distinguish between empty and non-empty voxels. Our method improves the 3D OD performance by 1.75 mAP points and 1.05 NDS on the challenging nuScenes dataset. Compared to existing self-supervised methods for automotive data, Voxel-MAE displays up to a 2× performance increase. Further, we show that by pretraining with Voxel-MAE, we require only 40% of the annotated data to outperform a randomly initialized equivalent. Code will be released.

show abstract

Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learning

Sheng

Shen

Xiao

2023

AAAI

View full text Add to dashboard Cite

We present a new self-supervised paradigm on point cloud sequence understanding. Inspired by the discriminative and generative self-supervised methods, we design two tasks, namely point cloud sequence based Contrastive Prediction and Reconstruction (CPR), to collaboratively learn more comprehensive spatiotemporal representations. Specifically, dense point cloud segments are first input into an encoder to extract embeddings. All but the last ones are then aggregated by a context-aware autoregressor to make predictions for the last target segment. Towards the goal of modeling multi-granularity structures, local and global contrastive learning are performed between predictions and targets. To further improve the generalization of representations, the predictions are also utilized to reconstruct raw point cloud sequences by a decoder, where point cloud colorization is employed to discriminate against different frames. By combining classic contrast and reconstruction paradigms, it makes the learned representations with both global discrimination and local perception. We conduct experiments on four point cloud sequence benchmarks, and report the results on action recognition and gesture recognition under multiple experimental settings. The performances are comparable with supervised methods and show powerful transferability.

show abstract

Masked Discrimination for Self-Supervised Learning on Point Clouds

Cited by 4 publications

References 0 publications

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Clouds

Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learning

Contact Info

Product

Resources

About