Temporal Coherence Methods in Real‐Time Rendering

Abstract

Nowadays, there is a strong trend towards rendering to higher‐resolution displays and at high frame rates. This development aims at delivering more detail and better accuracy, but it also comes at a significant cost. Although graphics cards continue to evolve with an ever‐increasing amount of computational power, the speed gain is easily counteracted by increasingly complex and sophisticated shading computations. For real‐time applications, the direct consequence is that image resolution and temporal resolution are often the first candidates to bow to the performance constraints (e.g. although full HD is possible, PS3 and XBox often render at lower resolutions). In order to achieve high‐quality rendering at a lower cost, one can exploit temporal coherence (TC). The underlying observation is that a higher resolution and frame rate do not necessarily imply a much higher workload, but a larger amount of redundancy and a higher potential for amortizing rendering over several frames. In this survey, we investigate methods that make use of this principle and provide practical and theoretical advice on how to exploit TC for performance optimization. These methods not only allow incorporating more computationally intensive shading effects into many existing applications, but also offer exciting opportunities for extending high‐end graphics applications to lower‐spec consumer‐level hardware. To this end, we first introduce the notion and main concepts of TC, including an overview of historical methods. We then describe a general approach, image‐space reprojection, with several implementation algorithms that facilitate reusing shading information across adjacent frames. We also discuss data‐reuse quality and performance related to reprojection techniques. Finally, in the second half of this survey, we demonstrate various applications that exploit TC in real‐time rendering.

<inline-formula><tex-math notation="LaTeX">$k^+$</tex-math><alternatives> <inline-graphic xlink:type="simple" xlink:href="vasilakis-ieq1-2417581.gif"/></alternatives></inline-formula>-buffer: An Efficient, Memory-Friendly and Dynamic <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives> <inline-graphic xlink:type="simple" xlink:href="vasilakis-ieq2-2417581.gif"/></alternatives></inline-formula>-buffer Fram

“…While our fragment outlier rejection mechanism is a key feature to our framework, additional research has to be conducted to alleviate its order-dependence performance nature. Additionally, fragment culling acceleration may be achieved by taking advantage of temporal coherence across adjacent frames [50]. Finally, our framework can be combined with an attention-based level-of-detail manager to downgrade fragment storage in areas that are expected to go unnoticed by an observer [51].…”

Efficient Hybrid Image Warping for High Frame-Rate Stereoscopic Rendering

“…the target image may contain artifacts caused by holes and folds. A survey of rendering systems exploiting temporal coherence by image warping was given by [22]. Furthermore, image warping has been used to generate post-processing effects such as motion blur and depth of field [16].…”

Temporal Video Filtering and Exposure Control for Perceptual Motion Blur

“…For CG-generated movies, we can modify the upsampling process and derive an UHFR sequence more easily by relying on existing temporal coherence methods [39]. In particular, given a physically-based renderer, which have become common in production rendering [40], [41], we can create a low quality UHFR video as input to our algorithm (Fig.…”