Deep Parametric Continuous Convolutional Neural Networks

Wang, Shenlong; Suo, Simon; Ma, Wei-Chiu; Pokrovsky, Andrei; Urtasun, Raquel

doi:10.1109/cvpr.2018.00274

Cited by 429 publications

(302 citation statements)

References 14 publications

Supporting

Mentioning

301

Contrasting

Unclassified

Order By: Relevance

“…PointCNN [7] explores convolution on point clouds and addresses the point ordering issue by permuting and weighting input points and features with the X -Conv operator. Besides, methods of [16,22,27,26,24] explore local context based on graphs.…”

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

“…KCNet [14] creates k-nearest neighbor graphs and applies kernel correlation to learn local structures over point neighborhood. PCCN [24] and PointConv [27] connect each point with its k-nearest neighbors and extend the convolution operation from regular grids to irregular point clouds by adaptively projecting the relative position of two points to a convolution weight. Compared to PCCN, PointConv additionally considers point distribution density.…”

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

“…In our work, we also propose a graph for point cloud processing, and yet focus particularly on exploring the semantic relation between points and their contextual neighbors for semantic segmentation through explicit edges. The key distinction of our method from other graph-based frameworks is that instead of fixing the graph and point resolution (e.g., PCCN [24] and KCNet [14]) or building independent graphs at each scale (e.g., PointConv [27], PointWeb [29] and ECC [16]), our graph is hierarchically constructed. We construct an edge branch, in which we fuse multi-scale point features and propagate edge features over multiple scales to enable longer distances of message passing hierarchically over edges without large memory overhead.…”

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

“…With meaningful edge features, we further feed edge features into each scale of the point branch to offer contextual information. To pass messages via edges, PointConv [27] and PCCN [24] adaptively learn weights from edges to fuse point features, while KCNet [14] defines a point-set kernel and kernel correlation to aggregate local features along edges. Different from these methods, our approach concatenates each point feature with the max-pooled corresponding edge features.…”

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

See 3 more Smart Citations

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

Jiang

Zhao

Liu³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

193

View full text Add to dashboard Cite

We achieve 3D semantic scene labeling by exploring semantic relation between each point and its contextual neighbors through edges. Besides an encoder-decoder branch for predicting point labels, we construct an edge branch to hierarchically integrate point features and generate edge features. To incorporate point features in the edge branch, we establish a hierarchical graph framework, where the graph is initialized from a coarse layer and gradually enriched along the point decoding process. For each edge in the final graph, we predict a label to indicate the semantic consistency of the two connected points to enhance point prediction. At different layers, edge features are also fed into the corresponding point module to integrate contextual information for message passing enhancement in local regions. The two branches interact with each other and cooperate in segmentation. Decent experimental results on several 3D semantic labeling datasets demonstrate the effectiveness of our work.

show abstract

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

Section: Point-based Deep Neural Networkmentioning

confidence: 99%

See 2 more Smart Citations

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

Jiang

Zhao

Liu³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

193

View full text Add to dashboard Cite

show abstract

“…These projections allow 2D convolution, but information is lost in the reduction to 2D, rendering these approaches unsuitable for some environments, especially complex or cluttered scenes. Recently, [22] and [23] propose direct convolution over point clouds by adjusting the kernel weights locally according to irregular point positions.…”

Section: A Deep Learning On Point Cloudsmentioning

confidence: 99%

LDLS: 3-D Object Segmentation Through Label Diffusion From 2-D Images

Wang

Chao

Wang

et al. 2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Object segmentation in 3D point clouds is a critical task for robots capable of 3D perception. Despite the impressive performance of deep learning-based approaches on object segmentation in 2D images, deep learning has not been applied nearly as successfully for 3D point cloud segmentation. Deep networks generally require large amounts of labeled training data, which are readily available for 2D images but are difficult to produce for 3D point clouds. In this paper, we present Label Diffusion Lidar Segmentation (LDLS), a novel approach for 3D point cloud segmentation which leverages 2D segmentation of an RGB image from an aligned camera to avoid the need for training on annotated 3D data. We obtain 2D segmentation predictions by applying Mask-RCNN to the RGB image, and then link this image to a 3D lidar point cloud by building a graph of connections among 3D points and 2D pixels. This graph then directs a semi-supervised label diffusion process, where the 2D pixels act as source nodes that diffuse object label information through the 3D point cloud, resulting in a complete 3D point cloud segmentation. We conduct empirical studies on the KITTI benchmark data set and on a mobile robot, demonstrating wide applicability and superior performance of LDLS compared to the previous state-of-the-art in 3D point cloud segmentation, without any need for either 3D training data or fine-tuning of the 2D image segmentation model.

show abstract

Deep Continuous Fusion for Multi-sensor 3D Object Detection

Liang

Yang

Wang

et al. 2018

Lecture Notes in Computer Science

Self Cite

713

473

View full text Add to dashboard Cite

In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization. Towards this goal, we design an end-to-end learnable architecture that exploits continuous convolutions to fuse image and LIDAR feature maps at different levels of resolution. Our proposed continuous fusion layer encode both discrete-state image features as well as continuous geometric information. This enables us to design a novel, reliable and efficient end-to-end learnable 3D object detector based on multiple sensors. Our experimental evaluation on both KITTI as well as a large scale 3D object detection benchmark shows significant improvements over the state of the art.

show abstract

Deep Parametric Continuous Convolutional Neural Networks

Cited by 429 publications

References 14 publications

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

LDLS: 3-D Object Segmentation Through Label Diffusion From 2-D Images

Deep Continuous Fusion for Multi-sensor 3D Object Detection

Contact Info

Product

Resources

About