K-Planes: Explicit Radiance Fields in Space, Time, and Appearance

Fridovich-Keil, Sara; Meanti, Giacomo; Warburg, Frederik; Recht, Benjamin; Kanazawa, Angjoo

doi:10.48550/arxiv.2301.10241

Cited by 3 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The core component of NFM is the Spatially Sparse Neural Fields (SSNF), a hybrid INR employing a pyramid of multi-resolution, spatially sparse feature grids to maintain longrange buffers of spatiotemporal velocity fields. We show that for this purpose, our SSNF offers improved accuracy, training speed, and memory efficiency compared to state-of-the-art representations (e.g., Instant NGP [Müller et al 2022], K-Planes [Fridovich-Keil et al 2023], and SIREN [Sitzmann et al 2020]), reducing the fitting error by over 70%. Leveraging this effective neural structure, we compute high-quality bidirectional flow maps by marching the SSNF forward and backward in time, and consolidate them with the impulse-based fluid simulation method to drastically reduce simulation errors by over 90% with respect to analytical solutions.…”

Section: Introductionmentioning

confidence: 91%

“…However, pure neural representations suffer from their considerable time cost, and follow-up works have focused on hybrid INRs featuring classical data structures like dense [Sun et al 2022] and sparse voxel grids [Chabra et al 2020;Liu et al 2020;Martel et al 2021;Takikawa et al 2021;. Recently, plane-based data structures [Chen et al 2022;Fridovich-Keil et al 2023] have also been leveraged to good effects. Most relevant to our work is Instant NGP [Müller et al 2022], which [Park et al 2021a,b;Pumarola et al 2021] and scene flow fields [Du et al 2021;Li et al , 2021bXian et al 2021].…”

Section: Implicit Neural Representationmentioning

confidence: 99%

“…We compare SSNF to three benchmarks: Instant NGP (INGP) [Müller et al 2022], KPlanes [Fridovich-Keil et al 2023], andSIREN [Sitzmann et al 2020], in fitting spatiotemporal fields in both 2D and 3D (spatial-wise). These benchmarks are carefully selected to cover a wide spectrum of INR paradigms.…”

Section: Validation Of Ssnfmentioning

confidence: 99%

See 2 more Smart Citations

Fluid Simulation on Neural Flow Maps

Deng,

Yu,

Zhang

et al. 2023

ACM Trans. Graph.

View full text Add to dashboard Cite

We introduce Neural Flow Maps, a novel simulation method bridging the emerging paradigm of implicit neural representations with fluid simulation based on the theory of flow maps, to achieve state-of-the-art simulation of in-viscid fluid phenomena. We devise a novel hybrid neural field representation, Spatially Sparse Neural Fields (SSNF), which fuses small neural networks with a pyramid of overlapping, multi-resolution, and spatially sparse grids, to compactly represent long-term spatiotemporal velocity fields at high accuracy. With this neural velocity buffer in hand, we compute long-term, bidirectional flow maps and their Jacobians in a mechanistically symmetric manner, to facilitate drastic accuracy improvement over existing solutions. These long-range, bidirectional flow maps enable high advection accuracy with low dissipation, which in turn facilitates high-fidelity incompressible flow simulations that manifest intricate vortical structures. We demonstrate the efficacy of our neural fluid simulation in a variety of challenging simulation scenarios, including leapfrogging vortices, colliding vortices, vortex reconnections, as well as vortex generation from moving obstacles and density differences. Our examples show increased performance over existing methods in terms of energy conservation, visual complexity, adherence to experimental observations, and preservation of detailed vortical structures.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Implicit Neural Representationmentioning

confidence: 99%

See 1 more Smart Citation

Fluid Simulation on Neural Flow Maps

Deng,

Yu,

Zhang

et al. 2023

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…Another category of approaches learn a time-varying deformation of 3D points into a static canonical scene (Pumarola et al, 2021;Park et al, 2021a;Tretschk et al, 2021;Park et al, 2021b). Some approaches accelerate NeRFs on dynamic scenes using explicit voxel grids (Fang et al, 2022) or tensor factorization (Cao & Johnson, 2023;Fridovich-Keil et al, 2023).…”

Section: Related Workmentioning

confidence: 99%

Text-To-4D Dynamic Scene Generation

Singer¹,

Sheynin²,

Polyak³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Serval studies adopt a hybrid representation [3,4,10,14,31,33,38,44,52,62] to speed up the reconstruction and rendering. They employ explicit data structures such as discrete voxel grids [14,52], decomposed tensors [3,4,13], hash maps [38], etc. to store features or spherical harmonics, enabling fast convergence and inference.…”

Section: Related Workmentioning

confidence: 99%

StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields

Liu¹,

Zhan²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

3D style transfer aims to render stylized novel views of a 3D scene with multi-view consistency. However, most existing work suffers from a three-way dilemma over accurate geometry reconstruction, high-quality stylization, and being generalizable to arbitrary new styles. We propose StyleRF (Style Radiance Fields), an innovative 3D style transfer technique that resolves the three-way dilemma by performing style transformation within the feature space of a radiance field. StyleRF employs an explicit grid of high-level features to represent 3D scenes, with which highfidelity geometry can be reliably restored via volume rendering. In addition, it transforms the grid features according to the reference style which directly leads to highquality zero-shot style transfer. StyleRF consists of two innovative designs. The first is sampling-invariant content transformation that makes the transformation invari-

show abstract

K-Planes: Explicit Radiance Fields in Space, Time, and Appearance

Cited by 3 publications

References 0 publications

Fluid Simulation on Neural Flow Maps

Fluid Simulation on Neural Flow Maps

Text-To-4D Dynamic Scene Generation

StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields

Contact Info

Product

Resources

About