BACON: Band-limited Coordinate Networks for Multiscale Scene Representation

Abstract

Coordinate-based networks have emerged as a powerful tool for 3D representation and scene reconstruction. These networks are trained to map continuous input coordinates to the value of a signal at each point. Still, current architectures are black boxes: their spectral characteristics cannot be easily analyzed, and their behavior at unsupervised points is difficult to predict. Moreover, these networks are typically trained to represent a signal at a single scale, and so naive downsampling or upsampling results in artifacts. We introduce band-limited coordinate networks (BACON), a network architecture with an analytical Fourier spectrum. BACON has predictable behavior at unsupervised points, can be designed based on the spectral characteristics of the represented signal, and can represent signals at multiple scales without explicit supervision. We demonstrate BA-CON for multiscale neural representation of images, radiance fields, and 3D scenes using signed distance functions and show that it outperforms conventional single-scale coordinate networks in terms of interpretability and quality.

MIP-plicits: Level of Detail Factorization of Neural Implicits Sphere Tracing

“…More exploration can also be done using other networks for LOD, both for the Multiscale ST and for attribute mapping. For example, [ 11,18] introduced multiplicative filter networks which allows LOD to be encoded in a single network. This network could be used in conjunction with MIP-plicits.…”

Progressive Multi-scale Light Field Networks

“…[40,37,38], real-time inference [17,42,33,64], improved quality [49], generalization across scenes [66,41], generative modelling [47,35,60], videos [24,57,55], sparse views [8,34,19], fast training [32,64], and even large-scale scenes [43,58,54]. Among NeRF research, Mip-NeRF [3,4] and BA-CON [27] share a similar goal to our work in offering a multi-scale representation to reduce aliasing. Mip-NeRF approximates the radiance across conical regions along the ray, cone-tracing, using integrated positional encoding (IPE).…”

“…With cone-tracing, Mip-NeRF reduces aliasing while also slightly reducing training time. BACON [27] provide a multi-scale scene representation through band-limited networks using Multiplicative Filter Networks [13] and multiple exit points. Each scale in BACON has band-limited outputs in Fourier space.…”

“…For radiance fields, Yang et al [61] develop Recursive-NeRF which grows a neural representa-tion with multi-stage training which is able to reduce NeRF rendering time. BACON [27] offers levels of detail for various scene representations including 2D images and radiance fields with different scales encoding different bandwidths in Fourier space. PINs [20] uses a progressive fourier encoding to improve reconstruction for scenes with a wide frequency spectrum.…”