Real-time temporal quality compensation technique for head mounted displays

Chang, Jung-Woo; Kang, Suk‐Ju; Seo, Min-Woo; Choi, Song-Woo; Lee, Sang-Lyn; Lee, Ho-Chul; Oh, Euiyeol; Baek, Jong-Sang

doi:10.1145/3145690.3145714

Cited by 2 publications

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was demonstrated in [44] that timewarp can be implemented using an integrated GPU. Chang et al [45] implemented timewarp in FPGA, thereby reducing the computational load of GPU. Timewarp can also be applied to scenes and objects separately [46].…”

Section: E Timewarpmentioning

confidence: 99%

Selective Timewarp Based on Embedded Motion Vectors for Interactive Cloud Virtual Reality

Nguyen

Kim

Son³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Interactive virtual reality (VR) services such as VR gaming require considerable computing power for rendering higher quality VR images at a high frame rate. Thus offloading such VR processing to a cloud or edge computing entity is promising, but is accompanied by long latency leads for a user to see an image of the past viewport, resulting in prohibitive motion sickness. Time warp is a technique for VR that warps a rendered image before scanning it out to the display to correct for the head movement occurring after the rendering and will play an important role in reducing the perceived latency in cloud/edge VR. However, time warp has to be applied to the image areas excluding the head-locked objects such as head-up display, menu bar, and notification that are intended to have a fixed position on a screen; otherwise, they are shown to judder. In this paper, we propose an algorithm that identifies head-locked objects in encoded VR frames at low computational load with no explicit information of the head-locked objects for selective time warp in the cloud/edge VR environments. First, we make a key observation from testbed experiments that motion vectors embedded in an encoded VR video stream are highly correlated with the user's head motion. Based on this finding, the head-locked object detection algorithm is designed to: 1) find a raw shape of each head-locked object from motion vectors embedded in frames and 2) identify the exact pixel-level head-locked region within a limited search area (along the boundary of each raw shape) by monitoring pixel color changes over frames. In order to achieve even lower computational load, the costly part of the algorithm is activated only when a new head-locked object appears. The experimental study demonstrates that the algorithm is able to detect multiple head-locked objects with zero pixel error under limited computational load.INDEX TERMS Virtual reality, cloud VR, timewarp, image reprojection, head-locked object, motion vector.

show abstract

Section: E Timewarpmentioning

confidence: 99%