Textured Mesh Quality Assessment: Large-Scale Dataset and Deep Learning-based Quality Metric

Abstract

Over the past decade, 3D graphics have become highly detailed to mimic the real world, exploding their size and complexity and making them subject to lossy processing operations that may degrade their visual quality. Thus, to ensure the best Quality of Experience (QoE), it is important to evaluate the visual quality to accurately drive the processing operation to find the right compromise between visual quality and data size. In this work, we evaluate the quality of textured 3D meshes. We first establish a large-scale quality assessment dataset, which includes 55 source models and over 343k distorted stimuli. Each model was characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of 5 types of distortions applied on the geometry and texture of the meshes. We then propose an approach to select a subset of challenging stimuli from our (large-scale) dataset that we annotate in a subjective experiment conducted in crowdsourcing. Leveraging our dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric to explore the influence of distortion interactions on the perceived quality of 3D graphics.

Multi-view stereo of an object immersed in a refractive medium

“…Source of the 3D model: LIRIS's datasets library. 54 Fig. 7 3D reconstruction of a graphosoma immersed in a cube of epoxy resin, seen from three angles, obtained by our RMVS solving method from 18 synthetic images such as those in Fig.…”

Subjective and objective quality assessment for volumetric video

“…Pioneering subjective quality tests involving meshes were conducted on still rendered images [33,34]; those two early studies both assessed the visual impact of simplification artifacts and concerned static and geometry-only meshes; the same applies in [35], which evaluated the impact of geometry compression. Subsequent passive interaction experiments considered meshes with color/texture attributes [36][37][38][39][40][41] or dynamic meshes [8,42,43]. These studies are detailed below.…”

“…It is worth mentioning that crowd-sourcing is quite faster to evaluate large datasets, yet the most time intensive task is building and designing the experimental framework (or setup) (user-friendly tool, control viewer environment, add screening test, etc.). Based on these findings, a largescale crowd-sourcing experiment was conducted to rate the perceived quality of the largest dataset of textured meshes to this date [41]. This dataset allowed to analyze the impact of the distortions and model characteristics (geometric and color complexity) on the perceived quality of textured meshes.…”

“…The only public dataset involving meshes with vertex colors is provided by Nehmé et al [39] and contains 480 distorted stimuli. For textured meshes, three datasets exist: [37], [8] and [41]. The first two datasets contain, respectively, 136 and 28 stimuli, whereas the latter contains more than 343k stimuli, of which 3000 (a generalized and challenging subset) are associated with mean opinion scores (MOS) derived from subjective experiments, and the rest with predicted quality scores (pseudo-MOS), making it the largest quality assessment dataset of textured meshes to date.…”

“…A more recent metric called graphics-LPIPS was proposed for assessing the quality of rendered 3D graphics [41]. The metric is an extension of the LPIPS metric (originally designed for images and perceptual similarity tasks) [134], which has been adapted to 3D graphics and quality assessment tasks based on DSIS.…”