3D visual saliency and convolutional neural network for blind mesh quality assessment

Abouelaziz, Ilyass; Chetouani, Aladine; Latecki, Longin Jan; Cherifi, Hocine

doi:10.1007/s00521-019-04521-1

Cited by 23 publications

(16 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We finally fine-tuned each model to adapt their weights to our context. It is worth noting that the target of each stacked patch was the subjective score of the whole distorted PC as commonly employed to estimate the quality of several multimedia content for 2D images [20,21] as well as for stereo images [22] and 3D meshes [23].…”

Section: Cnn Models and Patch Quality Indexesmentioning

confidence: 99%

Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

Chetouani

Quach

Valenzise

et al. 2021

2021 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

Point cloud (PC) quality assessment is of fundamental importance to enable the efficient processing, coding and transmission of 3D data for virtual/augmented reality, autonomous driving, cultural heritage, etc. The quality metrics proposed so far aim at quantifying the distortion in the PC geometry and/or attributes with respect to a reference pristine point cloud, using simple features extracted by the points. In this work, we target instead a blind (no-reference) scenario in which the original point cloud is not available. In addition, we learn features from data using deep neural networks. Given the limited availability of subjectively annotated datasets of corrupted point clouds, and the consequent difficulty to learn in an end-to-end fashion PC quality features, in this work we use instead a two-step procedure. First, we extract from local patches three relevant low-level features which have been commonly used in other PC quality metrics, i.e., geometric distance, local curvature and luminance values. Afterwards, we employ a deep neural network to learn, from these lowlevel features, a mapping to the PC ground truth mean opinion score. Our results on two state-of-the-art PC quality datasets show the potential of the proposed approach. The code is

show abstract

Section: Cnn Models and Patch Quality Indexesmentioning

confidence: 99%

Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

Chetouani

Quach

Valenzise

et al. 2021

2021 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To detect these tactile points, deep learning associated to learningto-rank methods were used on collected crowdsourcing data. Always based on saliency, Abouelaziz et al [35,36] proposed a no-reference deep learning approach where the convolutional network is fed with 2D saliency patches in order to estimate the visual quality of a 3D mesh.…”

Section: D Mesh Visual Saliency Approaches In the State-of-the-artmentioning

confidence: 99%

A Genetically Based Combination of Visual Saliency and Roughness for FR 3D Mesh Quality Assessment: A Statistical Study

Nouri

Charrier

Lézoray

2020

The Computer Journal

View full text Add to dashboard Cite

In this paper, we present a full-reference quality assessment metric based on the information of visual saliency. The saliency information is provided under the form of degrees associated to each vertex of the surface mesh. From these degrees, statistical attributes reflecting the structures of the reference and distorted meshes are computed. These are used by four comparisons functions genetically optimized that quantify the structure differences between a reference and a distorted mesh. We also present a statistical comparison study of six full-reference quality assessment metrics for 3D meshes. We compare the objective metrics results with humans subjective scores of quality considering the 3D meshes in one hand and the distorsion types in the other hand. Also, we show which metrics are statistically superior to their counterparts. For these comparisons we use the Spearman Rank Ordered Correlation Coefficient and the hypothetic test of Student (ttest). To attest the pertinence of the proposed approach, a comparison with a ground truth saliency and an application associated to the assessment of the visual rendering of smoothing algorithms are presented. Experimental results show that the proposed metric is very competitive with the state-of-the-art.

show abstract

“…image [3], [17], [18], stereo [62], 3D meshes [? ], [1], etc. ), remote sensing [30], watermarking [34], map viewing [51] [5], indoor localization [29], perception [14], image enhancement [12], [19], healthcare [38] among many others.…”

Section: Introductionmentioning

confidence: 99%

SATSal: A Multi-Level Self-Attention Based Architecture for Visual Saliency Prediction

et al. 2022

Self Cite

View full text Add to dashboard Cite

Human visual Attention modelling is a persistent interdisciplinary research challenge, gaining new interest in recent years mainly due to the latest developments in deep learning. That is particularly evident in saliency benchmarks. Novel deep learning-based visual saliency models show promising results in capturing high-level (top-down) human visual attention processes. Therefore, they strongly differ from the earlier approaches, mainly characterised by low-level (bottom-up) visual features. These developments account for innate human selectivity mechanisms that are reliant on both high-and low-level factors. Moreover, the two factors interact with each other. Motivated by the importance of these interactions, in this project, we tackle visual saliency modelling holistically, examining if we could consider both high-and low-level features that govern human attention. Specifically, we propose a novel method SAtSal (Self-Attention Saliency). SAtSal leverages both high and low-level features using a multilevel merging of skip connections during the decoding stage. Consequently, we incorporate convolutional self-attention modules on skip connection from the encoder to the decoder network to properly integrate the valuable signals from multilevel spatial features. Thus, the self-attention modules learn to filter out the latent representation of the salient regions from the other irrelevant information in an embedded and joint manner with the main encoder-decoder model backbone. Finally, we evaluate SAtSal against various existing solutions to validate our approach, using the well-known standard saliency benchmark MIT300. To further examine SAtSal's robustness on other image types, we also evaluate it on the Le-Meur saliency painting benchmark.

show abstract

3D visual saliency and convolutional neural network for blind mesh quality assessment

Cited by 23 publications

References 40 publications

Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

Deep Learning-Based Quality Assessment Of 3d Point Clouds Without Reference

A Genetically Based Combination of Visual Saliency and Roughness for FR 3D Mesh Quality Assessment: A Statistical Study

SATSal: A Multi-Level Self-Attention Based Architecture for Visual Saliency Prediction

Contact Info

Product

Resources

About