Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

Chetouani, Aladine; Quach, Maurice; Valenzise, Giuseppe; Dufaux, Frédéric

doi:10.1109/icmew53276.2021.9455967

Cited by 27 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These results might be explained by the fact that AlexNet is a shallow model and thus cannot extract enough relevant features. For ResNet based models, it seems that the skip connections introduce a kind of redundancy that is not relevant for the quality task [56], [57]. However, it is giving competitive results compared to the state-of-the-art methods on ICIP2020 database.…”

Section: ) Impact Of the Fine-tuned Cnn Networkmentioning

confidence: 99%

No-Reference 3D Point Cloud Quality Assessment Using Multi-View Projection and Deep Convolutional Neural Network

et al. 2023

Self Cite

View full text Add to dashboard Cite

Digital representation of 3D content in the form of 3D point clouds (PC) has gained increasing interest and has emerged in various computer vision applications. However, various degradation may appear on the PC during acquisition, transmission, or treatment steps in the 3D processing pipeline. Therefore, several Full-Reference, Reduced-Reference, and No-Reference metrics have been proposed to estimate the visual quality of PC. However, Full-Reference and Reduced-Reference metrics require reference information, which is not accessible in real-world applications, and No-Reference metrics still lack precision in evaluating the PC quality. In this context, we propose a novel deep learning-based method for No-Reference Point Cloud Quality Assessment (NR-PCQA) that aims to automatically predict the perceived visual quality of the PC without using the reference content. More specifically, in order to imitate the human visual system during the PC quality evaluation that captures the geometric and color degradation, we render the PC into different 2D views using a perspective projection. Then, the projected 2D views are divided into patches that are fed to a Convolutional Neural Network (CNN) to learn sophisticated and discriminative visual quality features for evaluating the local quality of each patch. Finally, the overall quality score of the PC is obtained by pooling the quality score patches. We conduct extensive experiments on three benchmark databases: ICIP2020, SJTU, and WPC, and we compare the proposed model to the existing Full-Reference, Reduced-Reference, and No-Reference state-of-the-art methods. Based on the experimental results, our proposed model achieves high correlations with the subjective quality scores and outperforms the state-of-the-art methods.INDEX TERMS Point cloud, quality assessment, point cloud rendering, convolutional neural network (CNN).

show abstract

Section: ) Impact Of the Fine-tuned Cnn Networkmentioning

confidence: 99%

No-Reference 3D Point Cloud Quality Assessment Using Multi-View Projection and Deep Convolutional Neural Network

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Liu et al [14] propose a learning-based no-reference PCQA network, named ResSCNN, using voxelization followed by sparse 3D-CNN. Chetouani et al [24] propose a model using traditional CNN networks (e.g., VGG [25]) that first extract three low-level features (i.e., geometric distance, local curvature, and luminance values) from local patches and feed them into CNN. Liu et al [15] propose PQA-Net using multi-view projection followed by 2D CNN and distortion type identification module, but it is not end-to-end trained and ineffective when superposed distortion exists.…”

Section: Related Workmentioning

confidence: 99%

GPA-Net:No-Reference Point Cloud Quality Assessment with Multi-task Graph Convolutional Network

Ziyu¹,

Yang²,

Rui³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the rapid development of 3D vision, point cloud has become an increasingly popular 3D visual media content. Due to the irregular structure, point cloud has posed novel challenges to the related research, such as compression, transmission, rendering and quality assessment. In these latest researches, point cloud quality assessment (PCQA) has attracted wide attention due to its significant role in guiding practical applications, especially in many cases where the reference point cloud is unavailable. However, current no-reference metrics which based on prevalent deep neural network have apparent disadvantages. For example, to adapt to the irregular structure of point cloud, they require preprocessing such as voxelization and projection that introduce extra distortions, and the applied grid-kernel networks, such as Convolutional Neural Networks, fail to extract effective distortion-related features. Besides, they rarely consider the various distortion patterns and the philosophy that PCQA should exhibit shifting, scaling, and rotational invariance. In this paper, we propose a novel no-reference PCQA metric named the Graph convolutional PCQA network (GPA-Net). To extract effective features for PCQA, we propose a new graph convolution kernel, i.e., GPAConv, which attentively captures the perturbation of structure and texture. Then, we propose the multi-task framework consisting of one main task (quality regression) and two auxiliary tasks (distortion type and degree predictions). Finally, we propose a coordinate normalization module to stabilize the results of GPAConv under shift, scale and rotation transformations. Experimental results on two independent databases show that GPA-Net achieves the best performance compared to the state-of-the-art no-reference PCQA metrics, even better than some full-reference metrics in some cases. The code is available at: https://github.com/Slowhander/GPA-Net.git

show abstract

“…However, the exploration of no-reference PCQA is relatively late due to the limitation of prior knowledge and database. Therefore, [8] split point cloud into multiple local patches and used low-level patch-wise features (e.g., geometric distance, local curvature) to train a convolution neural network (CNN). [18] proposed to use multiview projection to realize point cloud data argumentation.…”

Section: Related Workmentioning

confidence: 99%

No-Reference Point Cloud Quality Assessment via Domain Adaptation

Yang¹,

Liu²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel no-reference quality assessment metric, the image transferred point cloud quality assessment (IT-PCQA), for 3D point clouds. For quality assessment, deep neural network (DNN) has shown compelling performance on no-reference metric design. However, the most challenging issue for no-reference PCQA is that we lack large-scale subjective databases to drive robust networks. Our motivation is that the human visual system (HVS) is the decision-maker regardless of the type of media for quality assessment. Leveraging the rich subjective scores of the natural images, we can quest the evaluation criteria of human perception via DNN and transfer the capability of prediction to 3D point clouds. In particular, we treat natural images as the source domain and point clouds as the target domain, and infer point cloud quality via unsupervised adversarial domain adaptation. To extract effective latent features and minimize the domain discrepancy, we propose a hierarchical feature encoder and a conditionaldiscriminative network. Considering that the ultimate purpose is regressing objective score, we introduce a novel conditional cross entropy loss in the conditional-discriminative network to penalize the negative samples which hinder the convergence of the quality regression network. Experimental results show that the proposed method can achieve higher performance than traditional no-reference metrics, even comparable results with full-reference metrics. The proposed method also suggests the feasibility of assessing the quality of specific media content without the expensive and cumbersome subjective evaluations.

show abstract

Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

Cited by 27 publications

References 24 publications

No-Reference 3D Point Cloud Quality Assessment Using Multi-View Projection and Deep Convolutional Neural Network

No-Reference 3D Point Cloud Quality Assessment Using Multi-View Projection and Deep Convolutional Neural Network

GPA-Net:No-Reference Point Cloud Quality Assessment with Multi-task Graph Convolutional Network

No-Reference Point Cloud Quality Assessment via Domain Adaptation

Contact Info

Product

Resources

About