Shading atlas streaming

Abstract

Streaming high quality rendering for virtual reality applications requires minimizing perceived latency. We introduce Shading Atlas Streaming (SAS), a novel object-space rendering framework suitable for streaming virtual reality content. SAS decouples server-side shading from client-side rendering, allowing the client to perform framerate upsampling and latency compensation autonomously for short periods of time. The shading information created by the server in object space is temporally coherent and can be efficiently compressed using standard MPEG encoding. Our results show that SAS compares favorably to previous methods for remote image-based rendering in terms of image quality and network bandwidth efficiency. SAS allows highly efficient parallel allocation in a virtualized-texture-like memory hierarchy, solving a common efficiency problem of object-space shading. With SAS, untethered virtual reality headsets can benefit from high quality rendering without paying in increased latency.

Tessellated Shading Streaming

“…Although the sample locations are moved, the connectivity of the tessellation does not change and thus the pattern shown in figure 3 still holds‐–which we will use for encoding. Note that this perspective adjustment is only possible when working on a triangle basis and not on patches [MVD∗ 18].…”

“…Similar to our approach Shading Atlas Streaming (SAS) [MVD∗18] also splits the rendering pipeline between a sever and client. While previous split rendering approaches tried to temporally or spatially augment shading on a client [CWC∗ 15], SAS reduces the geometry on the client and performs all complex shading on the server.…”

“…The atlas is transmitted to the client, which renders novel views from this data until new shading information arrives. Concurrently with our work, Shading Atlas Streaming (SAS) [MVD∗ 18], a similar approach has been published, underlining the importance of the topic.…”

“…Thus, streaming approaches must carefully manage texture space. Gathering shading in a predefined rectangular packing [MVD∗ 18], achieves good texture space utilization, but creates subpar sample distributions which either lead to high bandwidth requirements or low quality.…”

“…We outperform traditional image‐based and object‐space shading strategies in terms of quality, speed, and memory. In comparison to SAS [MVD∗ 18], we do not require preprocessing, achieve better quality, and avoid artifacts for slanted triangles. Our end‐to‐end implementation shows that our approach even works well on an untethered headset using an off‐the‐shelf smartphone.…”