Compression of Sparse and Dense Dynamic Point Clouds—Methods and Standards

Cao, Chun; Preda, Marius; Zakharchenko, Vladyslav; Jang, Euee S.; Zaharia, Titus

doi:10.1109/jproc.2021.3085957

Cited by 42 publications

(23 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A complete survey of point cloud compression solutions can be found here [10]. In our work, we elected to adopt the MPEG Anchor that was used to evaluate the Call for Proposals for the MPEG standardization efforts in point cloud compression [43], and the MPEG standard for dynamic point clouds V-PCC [22], as they have both been widely used in quality evaluation campaigns in the literature.…”

Section: Point Cloud Compressionmentioning

confidence: 99%

On the impact of VR assessment on the Quality of Experience of Highly Realistic Digital Humans

Viola¹,

Subramanyam²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Fuelled by the increase in popularity of virtual and augmented reality applications, point clouds have emerged as a popular 3D format for acquisition and rendering of digital humans, thanks to their versatility and real-time capabilities. Due to technological constraints and real-time rendering limitations, however, the visual quality of dynamic point cloud contents is seldom evaluated using virtual and augmented reality devices, instead relying on prerecorded videos displayed on conventional 2D screens. In this study, we evaluate how the visual quality of point clouds representing digital humans is affected by compression distortions. In particular, we compare three different viewing conditions based on the degrees of freedom that are granted to the viewer: passive viewing (2DTV), head rotation (3DoF), and rotation and translation (6DoF), to understand how interacting in the virtual space affects the perception of quality. We provide both quantitative and qualitative results of our evaluation involving 78 participants, and we make the data publicly available. To the best of our knowledge, this is the first study evaluating the quality of dynamic point clouds in virtual reality, and comparing it to traditional viewing settings. Results highlight the dependency of visual quality on the content under test, and limitations in the way current data sets are used to evaluate compression solutions. Moreover, influencing factors in quality evaluation in VR, and shortcomings in how point cloud This work is funded by the European Commission H2020 program, under the grant agreement 762111, VRTogether, http://vrtogether.eu/.

show abstract

Section: Point Cloud Compressionmentioning

confidence: 99%

On the impact of VR assessment on the Quality of Experience of Highly Realistic Digital Humans

Viola¹,

Subramanyam²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In general, the Binary Cross-Entropy (BCE) between estimated occupancy probability and real occupancy symbol is used in training, i.e., (6) where o(k) represents the real occupancy symbol (1 for POV or 0 for NOV), p = p MP-POV is the probability that this k-th MP-POV is POV, and thus (1-p) is the probability of being the NOV.…”

Section: Loss Functionsmentioning

confidence: 99%

“…Either uniform voxel or octree representation lets us leverage available spatial neighbors to construct a more precise context for entropy modeling. For example, in recentlyapproved ISO/IEC MPEG (Moving Picture Experts Group) Geometry-based PCC (G-PCC) standard [1], [6], the conditional probability of current POV is approximated using neighbors specified by parent-child connections in an octree presentation through a set of heuristic rules.…”

Section: Introductionmentioning

confidence: 99%

“…Future Prospects. After the conclusion of the G-PCC [6], international standardization groups, such as the ISO/IEC MPEG, and ISO/IEC JPEG (Joint Photographic Experts Group) are actively pursuing the next-generation point cloud compression using learning-based approaches [19], [20], which urges a solution with better coding efficiency, affordable complexity, and robust model generalization at the same time. Towards this goal, a more efficient 3D representation method and a better spatial neighbor utilization mechanism for adaptive context modeling are highly desired.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sparse Tensor-based Multiscale Representation for Point Cloud Geometry Compression

Wang¹,

Ding²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

This study develops a unified Point Cloud Geometry (PCG) compression method through Sparse Tensor Processing (STP) based multiscale representation of voxelized PCG, dubbed as the SparsePCGC. Applying the STP reduces the complexity significantly because it only performs the convolutions centered at Most-Probable Positively-Occupied Voxels (MP-POV). And the multiscale representation facilitates us to compress scale-wise MP-POVs progressively. The overall compression efficiency highly depends on the approximation accuracy of occupancy probability of each MP-POV, e.g., p MP-POV . Thus, we design the Sparse Convolution based Neural Networks (SparseCNN) consisting of sparse convolutions and voxel re-sampling to extensively exploit priors. We then develop the SparseCNN based Occupancy Probability Approximation (SOPA) model to estimate the p MP-POV s in a single-stage manner only using the cross-scale prior or in multi-stage by step-wisely utilizing autoregressive neighbors. Besides, we also suggest the SparseCNN based Local Neighborhood Embedding (SLNE) to characterize the local spatial variations as the feature attribute to improve the SOPA. Our unified approach shows the state-of-art performance in both lossless and lossy compression modes across a variety of datasets including the dense PCGs (8iVFB, Owlii) and the sparse LiDAR PCGs (KITTI, Ford) when compared with the MPEG G-PCC and other popular learning-based compression schemes. Furthermore, the proposed method presents lightweight complexity due to point-wise computation, and tiny storage desire because of model sharing across all scales. We make all materials publicly accessible at https://github.com/NJUVISION/SparsePCGC for reproducible research.

show abstract

“…Existing literature reviews provide an overview of MPEG standardization efforts (Schwarz et al, 2018;Graziosi et al, 2020;Cao et al, 2021), a classification taxonomy (Pereira et al, 2020) and an analysis based on the coding dimensionality (Cao et al, 2019). In this review, we will provide a general overview of PCC approaches with a focus on deep learning-based methods.…”

mentioning

confidence: 99%

Survey on Deep Learning-Based Point Cloud Compression

Quach¹,

Pang²,

Tian³

et al. 2022

Front. Signal Process.

View full text Add to dashboard Cite

Point clouds are becoming essential in key applications with advances in capture technologies leading to large volumes of data. Compression is thus essential for storage and transmission. In this work, the state of the art for geometry and attribute compression methods with a focus on deep learning based approaches is reviewed. The challenges faced when compressing geometry and attributes are considered, with an analysis of the current approaches to address them, their limitations and the relations between deep learning and traditional ones. Current open questions in point cloud compression, existing solutions and perspectives are identified and discussed. Finally, the link between existing point cloud compression research and research problems to relevant areas of adjacent fields, such as rendering in computer graphics, mesh compression and point cloud quality assessment, is highlighted.

show abstract

Compression of Sparse and Dense Dynamic Point Clouds—Methods and Standards

Cited by 42 publications

References 56 publications

On the impact of VR assessment on the Quality of Experience of Highly Realistic Digital Humans

On the impact of VR assessment on the Quality of Experience of Highly Realistic Digital Humans

Sparse Tensor-based Multiscale Representation for Point Cloud Geometry Compression

Survey on Deep Learning-Based Point Cloud Compression

Contact Info

Product

Resources

About