Mesh Saliency Detection Using Convolutional Neural Networks

Abstract

Mesh saliency has been widely considered as the measure of visual importance of certain parts of 3D geometries, distinguishable from their surroundings, with respect to human visual perception. This work is based on the use of convolutional neural networks to extract saliency maps fo large and dense 3D scanned models. The network is trained with saliency maps constructed with a fusion spectral and geometrical analysis generated measures. Extensive evaluation studies carried out, include visual perception evaluation, simplification and compression use cases. As a result, they verify the superiority of our approach as compared to other state-of-theart approaches. Furthermore, performance experiments indicate that CNN-based saliency extraction method is much faster in large and dense geometries allowing its application in low-latency and energy-efficient systems. Index Terms Saliency mapping extraction, Convolutional Neural Network, saliency features of 3D objects.

3D Visual Saliency: An Independent Perceptual Measure or A Derivative of 2D Image Saliency?

“…CfS-CNN [24]) by 126% and 23% in terms of LCC and AUC, respectively. The quantitative results indicate that 1) 3D visual saliency that predicts human visual attention on 3D surfaces might be perceptually related to 2D image saliency and categorical information of 3D objects, and 2) our method that combines the two types Spectral Processing [8], Point Clustering [13], Salient Regions [9], Hilbert-CNN [40], RPCA [37], CfS-CNN [24], the proposed MIMO-GAN-CRF and the ground-truth fixation maps provided by the 3DVA dataset [12]. Comparative results of more objects are available in the supplementary material.…”

“…The two methods avoided the training relying on vertex-level saliency annotations but were not evaluated with eye fixation ground truth. Nousias et al [40] trained a CNN to detect mesh saliency using pseudo ground truth generated by the handcrafted approach proposed in [37], which does not perform well at predicting real human fixations according to our experimental results.…”

“…Dev. of LCC ↓ Multi-Scale Gaussian [3] 0.131 0.265 Diffusion Wavelets [58] 0.088 0.222 Spectral Processing [8] 0.078 0.253 Point Clustering [13] 0.132 0.300 Salient Regions [9] 0.215 0.245 Hilbert-CNN [40] 0.113 0.267 RPCA [37] 0.199 0.251 CfS-CNN [24] 0 of knowledge via a GAN framework for detecting 3D visual saliency is computationally effective.…”

Deep 3D Geometric Saliency Estimation from Light Field Images

“…consisting of the k geometrical nearest vertices of vertex v i , estimated by using the k nearest neighbors (k-nn) algorithm (typically, we set k = 25). These patches are utilized to create n matrices N i ∈ R (k+1)×3 , consisting of the k + 1 corresponding normals [26]:…”

A Saliency Aware CNN-Based 3D Model Simplification and Compression Framework for Remote Inspection of Heritage Sites

“…A recent study employs CNNs to extract saliency maps on 3D meshes utilizing a multi-view setup [33]. Nousias et al [34] employed a CNN based mesh saliency extraction approach, employing a 3D geometric patch descriptor to classify faces into four classes. Greater class indices correspond to higher saliency values.…”