Point2color: 3D Point Cloud Colorization Using a Conditional Generative Network and Differentiable Rendering for Airborne LiDAR

Shinohara, Takayuki; Xiu, Haoyi; Matsuoka, M.

doi:10.1109/cvprw53098.2021.00117

Cited by 13 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, the latest work Point2Color [31] has a certain influence on the task of colourisation of point clouds. Point2color targets the point cloud colourisation of scenarios, where our previous work PCCN [12] is referred to as the first two work of point cloud colourisation.…”

Section: Methodsmentioning

confidence: 99%

“…Besides, we also evaluate PCCN-RE with other methods in the metric of SSIM to supplement the experiment on Table 5, which shows that our method achieves the best performance in a majority of objects. In recent years, the latest work Point2Color [31] has a certain influence on the task of colourisation of point clouds. Point2color targets the point cloud colourisation of scenarios, where our previous work PCCN [12] is referred to as the first two work of point cloud colourisation.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…PCCN-RE is proposed to complete this imperfection. Point2Color [31] designed a PointNet++-based cGANs to colourise point clouds in airborne 3-D point clouds. Point2color employs three modules: a discriminator to judge the fake colour of point clouds, a differentiable renderer that converts the coloured points into 2-D images, and another discriminator to judge fake colour in 2D images, which requires additional information computation and computation time.…”

Section: Colourisation Of the Point Cloudmentioning

confidence: 99%

See 2 more Smart Citations

PCCN‐RE: Point cloud colourisation network based on relevance embedding

Wang

Liu

2022

IET Computer Vision

View full text Add to dashboard Cite

The colour of the point cloud provides abundant information for perception tasks such as autonomous driving and virtual reality. However, few prior work studied the automatic colourisation of colourless point clouds. In this paper, the authors propose a novel method named as Point cloud Colorization Network based on Relevance Embedding (PCCN-RE) relying on three structures: a relevance embedding structure that efficiently captures local information through the calculation of a covariance matrix within nearby points; a weighted pooling structure designed to facilitate the fusion of features; an enhanced spatial transform network structure that keeps the invariance of input point clouds. On the ShapeNetCore dataset, our PCCN-RE generates more authentic colour than state-of-the-art methods for colourless point clouds and achieves the highest results by obtaining a Peak Signal to Noise Ratio of 9.40 and a Structural Similarity Index of 0.62.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Colourisation Of the Point Cloudmentioning

confidence: 99%

See 1 more Smart Citation

PCCN‐RE: Point cloud colourisation network based on relevance embedding

Wang

Liu

2022

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…Point2Color [ 12 ] performs point cloud colorization to estimate the color information of each point from a point cloud with no color information. The network uses PointNet++ for coloring.…”

Section: Related Workmentioning

confidence: 99%

Colored Point Cloud Completion for a Head Using Adversarial Rendered Image Loss

2022

View full text Add to dashboard Cite

Recent advances in depth measurement and its utilization have made point cloud processing more critical. Additionally, the human head is essential for communication, and its three-dimensional data are expected to be utilized in this regard. However, a single RGB-Depth (RGBD) camera is prone to occlusion and depth measurement failure for dark hair colors such as black hair. Recently, point cloud completion, where an entire point cloud is estimated and generated from a partial point cloud, has been studied, but only the shape is learned, rather than the completion of colored point clouds. Thus, this paper proposes a machine learning-based completion method for colored point clouds with XYZ location information and the International Commission on Illumination (CIE) LAB (L*a*b*) color information. The proposed method uses the color difference between point clouds based on the Chamfer Distance (CD) or Earth Mover’s Distance (EMD) of point cloud shape evaluation as a color loss. In addition, an adversarial loss to L*a*b*-Depth images rendered from the output point cloud can improve the visual quality. The experiments examined networks trained using a colored point cloud dataset created by combining two 3D datasets: hairstyles and faces. Experimental results show that using the adversarial loss with the colored point cloud renderer in the proposed method improves the image domain’s evaluation.

show abstract

“…PCCN [15] and [4] use a Generative Adversarial Networks (GAN) to colorize the point cloud but are limited to single objects. More recently, the Point2color [23] method, also based on a GAN, allows for the coloring of Airborne LiDAR scenes. However, this method uses two discriminators, one working directly on the cloud and the other on a projected image of the airborne view of the scene.…”

Section: B Point Cloud Coloringmentioning

confidence: 99%

Reducing domain shift in synthetic data augmentation for semantic segmentation of 3D point clouds

Cazorla¹,

Poinel²,

Papadakis

et al. 2022

2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

View full text Add to dashboard Cite

The use of deep learning in semantic segmentation of point clouds enables a drastic improvement of segmentation precision. However, available datasets are restrained to a few applications with limited applicability to other fields. Using synthetic and real data can alleviate the burden of creating a dedicated dataset at the cost of domain-shift that is mostly addressed during training, while treating the problem directly on the data has been less explored. Towards this goal, two methods to alleviate domain shift are proposed, firstly by enhanced generation and sampling of synthetic data and secondly by leveraging color information of unlabeled point clouds to color synthetic, uncoloured data. Obtained results confirm their usefulness in improving semantic segmentation result (+3.43 into mIoU for a network trained on S3DIS zone 1). More importantly, the devised coloring method shows the ability of a point-based network to link color information with recurrent geometric features. Finally, the presented methods are able to bridge the domain-shift gap even in cases where inclusion of raw synthetic data during training impedes learning.

show abstract

Point2color: 3D Point Cloud Colorization Using a Conditional Generative Network and Differentiable Rendering for Airborne LiDAR

Cited by 13 publications

References 31 publications

PCCN‐RE: Point cloud colourisation network based on relevance embedding

PCCN‐RE: Point cloud colourisation network based on relevance embedding

Colored Point Cloud Completion for a Head Using Adversarial Rendered Image Loss

Reducing domain shift in synthetic data augmentation for semantic segmentation of 3D point clouds

Contact Info

Product

Resources

About