Neural Residual Flow Fields for Efficient Video Representations

Daniel, Rho,; Cho, Jun Woo; Ko, Jong Hwan; Park, Eunbyung

doi:10.1007/978-3-031-26284-5_28

Cited by 8 publications

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides these tasks, NFs have been utilized in computer graphics and vision for image [KD18,WZY22] and video [KBE * 19] generation, compression [HDGS20] ] for more comprehensive reviews of these representations for neural rendering and computer vision. Besides their effectiveness for rendering, INRs have also been used for image [SPY * 22] and video [RCKP22] compression. We build upon these findings to design an efficient yet expressive INR for environment maps, as described in Section 5.…”

Section: Neural Sampling and Representationsmentioning

confidence: 99%

NEnv: Neural Environment Maps for Global Illumination

Rodríguez-Pardo

Fabre

Garcés

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Environment maps are commonly used to represent and compute far‐field illumination in virtual scenes. However, they are expensive to evaluate and sample from, limiting their applicability to real‐time rendering. Previous methods have focused on compression through spherical‐domain approximations, or on learning priors for natural, day‐light illumination. These hinder both accuracy and generality, and do not provide the probability information required for importance‐sampling Monte Carlo integration. We propose NEnv, a deep‐learning fully‐differentiable method, capable of compressing and learning to sample from a single environment map. NEnv is composed of two different neural networks: A normalizing flow, able to map samples from uniform distributions to the probability density of the illumination, also providing their corresponding probabilities; and an implicit neural representation which compresses the environment map into an efficient differentiable function. The computation time of environment samples with NEnv is two orders of magnitude less than with traditional methods. NEnv makes no assumptions regarding the content (i.e. natural illumination), thus achieving higher generality than previous learning‐based approaches. We share our implementation and a diverse dataset of trained neural environment maps, which can be easily integrated into existing rendering engines.

show abstract

Section: Neural Sampling and Representationsmentioning

confidence: 99%