Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering

Stengel, Michael; Grogorick, Steve; Eisemann, Martin; Magnor, Marcus

doi:10.1111/cgf.12956

Cited by 62 publications

(76 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system renders only the region centered on the gaze direction at the highest resolution, and uses larger pixels in the user's peripheral vision. Another approach is to include foveated rendering into a deferred shading pipeline by shading only some pixels, and by interpolating results for the remainder of the pixels [9,10].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Instantaneous foveated preview for progressive Monte Carlo rendering

et al. 2018

View full text Add to dashboard Cite

Progressive rendering, for example Monte Carlo rendering of 360• content for virtual reality headsets, is a time-consuming task. If the 3D artist notices an error while previewing the rendering, they must return to editing mode, make the required changes, and restart rendering. We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist's points of interest. The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye, unlike the piecewise linear models used in previous work. In a comprehensive user study, the perceived convergence of our proposed method was 10 times faster than that of a conventional preview, and often appeared to be instantaneous. In addition, the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch, compared to conventional rendering methods that had already generated image content in those areas.

show abstract

Section: Related Workmentioning

confidence: 99%

“…However, they are not as accurate in the peripheral parts of vision [8]. A simplified version uses a linear fall-off between the maximum and the minimum sampling probability [9][10][11], or even a static probability with respect to gaze direction [14].…”

Section: Related Workmentioning

confidence: 99%

Instantaneous foveated preview for progressive Monte Carlo rendering

et al. 2018

View full text Add to dashboard Cite

show abstract

“…However, they are not as accurate on the peripheral parts of the vision [Guenter et al 2012]. A further simplified version is to use a linear fall-off between full detail and the minimum sampling probability [Stengel et al 2016;Vaidyanathan et al 2014;Weier et al 2016], or even a static probability with respect to eye tracking [Pohl et al 2016].…”

Section: Related Workmentioning

confidence: 99%

Foveated instant preview for progressive rendering

Koskela

Immonen

Viitanen

et al. 2017

SIGGRAPH Asia 2017 Technical Briefs

View full text Add to dashboard Cite

Progressive rendering, for example Monte Carlo rendering of 360 • content for virtual reality headsets, is a time-consuming task. If the 3D artist notices an error while previewing the rendering, he or she must return to editing mode, do the required changes, and restart rendering. Restart is required because the rendering system cannot know which pixels are affected by the change. We propose the use of eye-tracking-based optimization to significantly speed up previewing the artist's points of interest. Moreover, we derive an optimized version of the visual acuity model, which follows the original model more accurately than previous work. The proposed optimization was tested with a comprehensive user study. The participants felt that preview with the proposed method converged instantly, and the recorded split times show that the preview is 10 times faster than conventional preview. In addition, the system does not have measurable drawbacks on computational performance.

show abstract

“…Such systems, however, have limited applicability as the user's exact focus cannot be predicted. However, in [SGEM16] a deferred shading system is introduced that creates a stochastic sampling pattern to adaptively select the rasterized fragments for shading based on the user's gaze -shading is the most computationally demanding part of modern image synthesis algorithms. An early foveated rendering system for volume rendering was developed by [LW90], using a multi-resolution technique to alter the resolution and sampling rates.…”

Section: Related Workmentioning

confidence: 99%

“…To still achieve an acceptable visual quality, this method needs to rely on specific anti-aliasing methods, limiting its applicability [SGEM16]. In [GFD ⇤ 12], rasterizing the image in three layers with different resolutions yields a speedup of 6.2 with only 7% of the pixels being rendered as the images are strongly undersampled.…”

Section: Benchmarksmentioning

confidence: 99%

Foveated Real‐Time Ray Tracing for Head‐Mounted Displays

Weier

Roth

Kruijff

et al. 2016

Computer Graphics Forum

View full text Add to dashboard Cite

a) Small foveal region with (r 0 = 5 , r 1 = 10 , p min = 0.01) (b) Medium foveal region with (r 0 = 10 , r 1 = 20 , p min = 0.05) (c) Full Renderer Figure 1: Images generated by using our foveated renderer showing the effect of different configurations for the foveal region, including an image that was rendered by ray tracing every pixel. AbstractHead-mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G-Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real-time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non-trivial static scenes for the Oculus DK2 HMD at 1182 ⇥ 1464 per eye within the the VSync limits without perceived visual differences.

show abstract

Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering

Cited by 62 publications

References 25 publications

Instantaneous foveated preview for progressive Monte Carlo rendering

Instantaneous foveated preview for progressive Monte Carlo rendering

Foveated instant preview for progressive rendering

Foveated Real‐Time Ray Tracing for Head‐Mounted Displays

Contact Info

Product

Resources

About