Visually Lossless Content and Motion Adaptive Shading in Games

Yang, Lei; Zhdan, Dmitry; Kilgariff, Emmett; Lum, Eric B.; Zhang, Yubo; Johnson, Matthew; Rydgård, Henrik

doi:10.1145/3320287

Cited by 14 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a comparative study, we picked two state-of-the-art models [Denes et al 2020] and [Yang et al 2019]. It should be noted that neither of these models (or any other model to our knowledge) is designed to predict our kind of data.…”

Section: Camojab Results and Comparisonmentioning

confidence: 99%

“…[Denes et al 2020] is a perceptually based model derived through extensive psychophysical experiments, however, it is content-independent. [Yang et al 2019] though content dependent, does not take display and viewing conditions into consideration. Furthermore, it is designed for a specific task to give quality scores for only full, half, and quarter shading resolution and not a continuous scale.…”

Section: Camojab Results and Comparisonmentioning

confidence: 99%

“…Our CaMoJAB metric accounts for most relevant factors of motion quality, including texture content, on-screen velocities, luminance, effective resolution, and display persistence. However, it does not model contrast masking [Mantiuk et al 2021], degradation of color, In comparison to Nvidia Adaptive Shading (NAS) [Yang et al 2019], CaMoJAB is calibrated in physical units (pixels-per-degree, degreesper-second, cd/m 2 ) and accounts for the display characteristics that affect motion quality perception, such as refresh rate, duty-cycle, and field-of-view. NAS may need to be recalibrated for a display of different size, brightness or pixel density than the one used by the authors.…”

Section: Limitations and Discussionmentioning

confidence: 99%

“…Both the metrics ignore the impact of content on motion quality. Yang et al [2019] proposed a metric for adaptive control of the VRS, called Nvidia Adaptive Shading (NAS), which considers both the screen content and the velocity. The metric analyses the screen content in the frequency domain and uses that information together with the motion vectors to classify each VRS tile into full, half, and quarter resolution shading.…”

Section: Models Of Motion Qualitymentioning

confidence: 99%

“…The recent introduction of VRS allowed for controlling the shading rates of each 16×16 pixel tiles enabling a plethora of optimizations. Vaidyanathan et al [2012] analyzed motion and defocus blur in the frequency domain to adaptively control the shading rate and Yang et al [2019] developed a similar framework better suited for VRS hardware. Drobot [2020] extends this idea to all platforms through their software based implementation of VRS.…”

Section: Motion Adaptive Renderingmentioning

confidence: 99%

See 4 more Smart Citations

Perceptual model for adaptive local shading and refresh rate

2021

View full text Add to dashboard Cite

Intel 2019]. All these works propose a dynamic quality control mechanism that allocates the rendering budget to those aspects of an image or animation that have the highest impact on the overall quality. In this work, we propose to control both the VRS state map and the refresh rate, based on all major factors affecting image quality: texture content, on-screen velocities, luminance, effective resolution, and display persistence. We build on the work of Denes et al. [2020], and extend the visual quality model to account for the important effect of texture content and VRS resolution. In contrast to that work, we offer local, rather than global, control of the resolution via VRS without the need for eye tracking.The key component of our Adaptive Local Shading and Refresh Rate (ALSaRR) method is a new Content-adaptive Metric of Judder, Aliasing and Blur (CaMoJAB) (Section 4). The metric is based on psychophysical models of contrast sensitivity with only a few parameters fitted to the data. We calibrate and validate our metric on various existing datasets as well as our new dataset collected by conducting a subjective quality experiment. The experiment measures the perceived loss of quality due to shading rate reduction under a large range of display refresh rates, resolutions, display persistence, luminance, contrast, and content velocity (Section 4.3). Our ALSaRR method uses the new metric to create per-texture quality functions, which are used for an approximate solution of the knapsack problem: maximize perceived quality for a given rendering budget (Section 5). Unlike the method of Yang et al. [2019], which controls VRS to avoid any visual loss regardless of the per-frame rendering cost, our goal is to find the best trade-off of spatio-temporal resolution under a limited rendering budget.The main contributions of our work 1 are:• Content-adaptive Metric of Judder, Aliasing and Blur (CaMo-JAB), derived from psychophysical models and calibrated on several datasets. • A dataset of motion quality for animations rendered with different shading rates, motion velocities, textures, refresh rates and display angular resolution (in pixels-per-degree), and persistence. • Adaptive Local Shading and Refresh Rate (ALSaRR) method for control of real-time rendering, which maximizes the quality of animation under a limited budget.

show abstract

Section: Camojab Results and Comparisonmentioning

confidence: 99%

Section: Camojab Results and Comparisonmentioning

confidence: 99%

Section: Limitations and Discussionmentioning

confidence: 99%

Section: Models Of Motion Qualitymentioning

confidence: 99%

Section: Motion Adaptive Renderingmentioning

confidence: 99%

See 3 more Smart Citations

Perceptual model for adaptive local shading and refresh rate

2021

View full text Add to dashboard Cite

show abstract

Scene‐content‐sensitive real‐time adaptive foveated rendering

Shen,

Chen,

2024

J Soc Info Display

View full text Add to dashboard Cite

In recent years, techniques for accelerating rendering by exploiting the limitations of the human visual system have become increasingly prevalent. The foveated rendering method significantly reduces the computational requirements during rendering by reducing image quality in peripheral regions. In this paper, we propose a scene‐content‐sensitive real‐time adaptive foveated rendering method. First, we pre‐render the three‐dimensional (3D) scene at a low resolution. Then, we utilize the low‐resolution pre‐rendered image as input to extract edge, local contrast, and color features. Subsequently, we generate a screen‐space region division map based on the gaze point position. Next, we calculate the visual importance of each 16 × 16 pixel tile based on edge, local contrast, color, and screen‐space region. We then map the visual importance to the shading rate to generate a shading rate control map for the current frame. Finally, we complete the rendering of the current frame based on variable rate shading technology. Experimental results demonstrate that our method effectively enhances the visual quality of images near the foveal region while generating high quality foveal region images. Furthermore, our method can significantly improve performance compared to per‐pixel shading method and existing scene‐content‐based foveated rendering methods.

show abstract

Visual Perception-Driven Dynamic Adaptive Shading

Shen

2024

2024 4th International Conference on Consumer Electronics and Computer Engineering (ICCECE)

View full text Add to dashboard Cite

Visually Lossless Content and Motion Adaptive Shading in Games

Cited by 14 publications

References 19 publications

Perceptual model for adaptive local shading and refresh rate

Perceptual model for adaptive local shading and refresh rate

Scene‐content‐sensitive real‐time adaptive foveated rendering

Visual Perception-Driven Dynamic Adaptive Shading

Contact Info

Product

Resources

About