Latency aware foveated rendering in unreal engine 4

Swafford, Nicholas T.; Cosker, Darren; Mitchell, Kim

doi:10.1145/2824840.2824863

Cited by 6 publications

(4 citation statements)

References 3 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We calculated the adjusted foveal diameter as described in [SCM15]: To deal with that, an intuitive solution is to enlarge the size of the foveal region.…”

Section: Foveated Stitchingmentioning

confidence: 99%

“…Thus, to compensate the delay, the foveal region diameter can be increased such that the true foveal field-of-view is always contained within the rendered foveal region, for some estimated overall system latency. We calculated the adjusted foveal diameter as described in [SCM15]:…”

Section: Foveated Stitchingmentioning

confidence: 99%

“…However, unlike the controllable latency environment in [GFD * 12], our setting in VR has the following challenges: (i) unstable latencies from different parts of the system, (ii) high computational cost of rendering high resolutional videos. Specifically, we set the proper foveal rendering diameters according to the estimated system latency [SCM15]. As the foveal regions of the users are estimated by tracking their gazes, when we lost track of that due to system delay, we assume the users direct their gazes to salient regions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High‐resolution 360 Video Foveated Stitching for Real‐time VR

Lee

Chen

et al. 2017

Computer Graphics Forum

View full text Add to dashboard Cite

Figure 1: In this paper, we present a novel high-resolution video stitching framework for real-time VR. We adapt image resolution and levelof-detail by estimating foveal region. The yellow circle indicates a highly-salient region thus rendered in high resolution, while red circle indicates less important region thus rendered in low resolution. The proposed foveated stitching greatly reduces the number of pixels to be processed and overall graphics computation. AbstractIn virtual reality (VR) applications, the contents are usually generated by creating a 360 • video panorama of a real-world scene. Although many capture devices are being released, getting high-resolution panoramas and displaying a virtual world in realtime remains challenging due to its computationally demanding nature. In this paper, we propose a real-time 360 • video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high-resolution panoramic video in real-time that can be streamed directly to the client. Our foveated stitching algorithm takes videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.

show abstract

“…We calculated the adjusted foveal diameter as described in [SCM15]: To deal with that, an intuitive solution is to enlarge the size of the foveal region.…”

Section: Foveated Stitchingmentioning

confidence: 99%

Section: Foveated Stitchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High‐resolution 360 Video Foveated Stitching for Real‐time VR

Lee

Chen

et al. 2017

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Moreover, they have calibrated and validated the model using several image quality and contrast databases. Swaffort et al further improve HDR‐VDP2 by providing an image metric for gaze‐contingent rendering quality [SCM15, SIGK*16], which adds measurements for the peripheral vision degradation at increasing eccentricities. The contrast sensitivity is thus decreased with visual eccentricity based on a tuned model of the CMF.…”

Section: Perceptual Modelsmentioning

confidence: 99%

Perception-driven Accelerated Rendering

Weier

Stengel

Roth

et al. 2017

Computer Graphics Forum

View full text Add to dashboard Cite

Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real-time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel (high dynamic range, wider color gamut/fidelity), increasing refresh rates (better motion depiction), and an increasing number of displayed views per pixel (stereo, multi-view, all the way to holographic or lightfield displays). These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.

show abstract

Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays

Bai

Wang³

et al. 2022

2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

Latency aware foveated rendering in unreal engine 4

Cited by 6 publications

References 3 publications

High‐resolution 360 Video Foveated Stitching for Real‐time VR

High‐resolution 360 Video Foveated Stitching for Real‐time VR

Perception-driven Accelerated Rendering

Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays

Contact Info

Product

Resources

About