Efficient Hybrid Image Warping for High Frame-Rate Stereoscopic Rendering

Abstract

Abstract-Modern virtual reality simulations require a constant high-frame rate from the rendering engine. They may also require very low latency and stereo images. Previous rendering engines for virtual reality applications have exploited spatial and temporal coherence by using image-warping to re-use previous frames or to render a stereo pair at lower cost than running the full render pipeline twice. However these previous approaches have shown artifacts or have not scaled well with image size. We present a new image-warping algorithm that has several novel contributions: an adaptive grid generation algorithm for proxy geometry for image warping; a low-pass hole-filling algorithm to address un-occlusion; and support for transparent surfaces by efficiently ray casting transparent fragments stored in per-pixel linked lists of an A-Buffer. We evaluate our algorithm with a variety of challenging test cases. The results show that it achieves better quality image-warping than state-of-the-art techniques and that it can support transparent surfaces effectively. Finally, we show that our algorithm can achieve image warping at rates suitable for practical use in a variety of applications on modern virtual reality equipment.

Time‐Warped Foveated Rendering for Virtual Reality Headsets

“…We recommend following Schollmeyer et al . 's [SSB*17] A‐buffer method if translucent materials are to be reused.…”

“…Schollmeyer et al . [SSB*17] improve on this concept with a tighter grid and support for transparency with an A‐buffer [Eng14] based data structure, where transparent, rasterized fragments are stored and subsequently ray‐traced from the new view. Without that, transparent fragments are difficult to warp as they are usually accumulated (alpha‐blended) and have no distinct depths.…”

“…A second class of artefacts is caused by mismatches in exact pixel locations for the reprojected data, i.e. small holes or inaccuracies in warping with coarse grid reprojection [WRK*16, SSB*17, DRE*10]. While some of the arising artefacts can be handled with cheap hole filling schemes, others must be solved by expensive redrawing.…”

Perceptual Rasterization for Head-mounted Display Image Synthesis

“…e main drawback of warping is that it su ers disocclusion artefacts. Some techniques can help ameliorate these, such as perceptually improved hole lling (Didyk et al 2010;Schollmeyer et al 2017). Alternatively the result can be improved by changing the images provided to the algorithm itself (Reinert et al 2016).…”

“…Image similarity is measured in terms of an adapted SSIM (Wang et al 2004) metric. It ignores all disoccluded pixels, i. e., it provides an upper bound on quality to what any hole lling, however sophisticated, could do (Didyk et al 2010;Schollmeyer et al 2017). For foveated comparisons, SSIM is computed for the 64×64 foveal pixels.…”