Tessellated Shading Streaming

Abstract

Presenting high‐fidelity 3D content on compact portable devices with low computational power is challenging. Smartphones, tablets and head‐mounted displays (HMDs) suffer from thermal and battery‐life constraints and thus cannot match the render quality of desktop PCs and laptops. Streaming rendering enables to show high‐quality content but can suffer from potentially high latency. We propose an approach to efficiently capture shading samples in object space and packing them into a texture. Streaming this texture to the client, we support temporal frame up‐sampling with high fidelity, low latency and high mobility. We introduce two novel sample distribution strategies and a novel triangle representation in the shading atlas space. Since such a system requires dynamic parallelism, we propose an implementation exploiting the power of hardware‐accelerated tessellation stages. Our approach allows fast de‐coding and rendering of extrapolated views on a client device by using hardware‐accelerated interpolation between shading samples and a set of potentially visible geometry. A comparison to existing shading methods shows that our sample distributions allow better client shading quality than previous atlas streaming approaches and outperforms image‐based methods in all relevant aspects.

SnakeBinning: Efficient Temporally Coherent Triangle Packing for Shading Streaming

“…The more slanted the triangle footprint, the more samples would be missed despite the conservative rasterization being enabled. This effect was already described by [HSS19b] and targeted by their oversampling strategy. The samples needed for bilinear interpolation-yet missed by conservative rasterization-are depicted as green samples in Figure 9 of [HSS19b].…”

“…The more advanced warping or upsampling technique employed by a remote rendering system, the more data they also need to transmit [NCO03,SH15] [HSS19b,HSS19a] perform shading completely on the server and pack shading into a transmitted atlas. The client renders novel views relying on straightforward texture mapping from a potentially visible set (PVS) of triangles, which is also transmitted by the server.…”

“…Typically one would use atomic operations to assign triangles to bins. However this would change the order of triangles in each bin completely every time binning is executed, even if the camera is still, much like in [HSS19b]. Obviously, this destroys the temporal coherency when encoding the atlas as a video frame and thus would strongly increase bandwidth.…”

“…While this allows to keep blocks at the same location in the atlas for multiple frames, sample distributions are typically sub-optimal, as they do not consider perspective effects and compromises between triangles combined into patches need to be made. Alternatively, every potentially visible triangle can be mapped separately into the shading atlas using Tessellated Shading Streaming (TSS) [HSS19b]. By employing tessellation and special treatment of slanted triangles, samples are distributed according to screen-space shape and perspective effects.…”

Glossy probe reprojection for interactive global illumination

“…While our work relies on sample reuse, our preprocessing is more exhaustive than these solutions. Object space shading can be used for remote rendering [Hladky et al 2019;Mueller et al 2018]. Scene triangles are packed in an atlas, while shading is updated on the server for visible triangles, and streamed to a local device using video compression.…”