Peter Bradbury

Bradbury, Chloe

doi:10.1136/bmj.n389

Cited by 17 publications

(27 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our deferred NeRF model is based on JAXNeRF [11], an implementation of NeRF in JAX [3]. As in NeRF, we apply a positional encoding [41] to positions and view di- 2.…”

Section: Implementation Detailsmentioning

confidence: 99%

Baking Neural Radiance Fields for Real-Time View Synthesis

Hedman¹,

Srinivasan²,

Mildenhall³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural volumetric representations such as Neural Radiance Fields (NeRF) have emerged as a compelling technique for learning to represent 3D scenes from images with the goal of rendering photorealistic images of the scene from unobserved viewpoints. However, NeRF's computational requirements are prohibitive for real-time applications: rendering views from a trained NeRF requires querying a multilayer perceptron (MLP) hundreds of times per ray. We present a method to train a NeRF, then precompute and store (i.e. "bake") it as a novel representation called a Sparse Neural Radiance Grid (SNeRG) that enables realtime rendering on commodity hardware. To achieve this, we introduce 1) a reformulation of NeRF's architecture, and 2) a sparse voxel grid representation with learned feature vectors. The resulting scene representation retains NeRF's ability to render fine geometric details and view-dependent appearance, is compact (averaging less than 90 MB per scene), and can be rendered in real-time (higher than 30 frames per second on a laptop GPU). Actual screen captures are shown in our video.

show abstract

“…Our deferred NeRF model is based on JAXNeRF [11], an implementation of NeRF in JAX [3]. As in NeRF, we apply a positional encoding [41] to positions and view di- 2.…”

Section: Implementation Detailsmentioning

confidence: 99%

Baking Neural Radiance Fields for Real-Time View Synthesis

Hedman¹,

Srinivasan²,

Mildenhall³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…ADDs (×10 6 ) MULs (×10 high area-to-volume ratio for the sake of power delivery and heat removal. Inability to co-locate various circuits and memories may force existence of long wires, so called buses.…”

Section: Layer Typementioning

confidence: 99%

“…A small variant of PokeBNN achieves 70.5% top-1 with 2.6 ACE, more than 3x reduction in cost; a larger PokeBNN achieves 75.6% top-1 with 7.8 ACE, more than 5% improvement in accuracy without increasing the cost. PokeBNN implementation in JAX/Flax [6,18] and reproduction instructions are open sourced. 2 * Work performed while at Google, equal contribution.…”

mentioning

confidence: 99%

PokeBNN: A Binary Pursuit of Lightweight Accuracy

Zhang¹,

Zhang²,

Lew³

2021

Preprint

View full text Add to dashboard Cite

Top-1 ImageNet optimization promotes enormous networks that may be impractical in inference settings. Binary neural networks (BNNs) have the potential to significantly lower the compute intensity but existing models suffer from low quality. To overcome this deficiency, we propose Poke-Conv, a binary convolution block which improves quality of BNNs by techniques such as adding multiple residual paths, and tuning the activation function. We apply it to ResNet-50 and optimize ResNet's initial convolutional layer which is hard to binarize. We name the resulting network family PokeBNN 1 . These techniques are chosen to yield favorable improvements in both top-1 accuracy and the network's cost. In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardware-and energy-inspired cost metric for quantized and binarized networks. We also identify a need to optimize an under-explored hyper-parameter controlling the binarization gradient approximation.We establish a new, strong state-of-the-art (SOTA) on top-1 accuracy together with commonly-used CPU64 cost, ACE cost and network size metrics. , the previous SOTA in BNNs, achieved a 70.5% top-1 accuracy with 7.9 ACE. A small variant of PokeBNN achieves 70.5% top-1 with 2.6 ACE, more than 3x reduction in cost; a larger PokeBNN achieves 75.6% top-1 with 7.8 ACE, more than 5% improvement in accuracy without increasing the cost. PokeBNN implementation in JAX/Flax [6,18] and reproduction instructions are open sourced. 2 * Work performed while at Google, equal contribution. 1 Poke/'p6kI/ is pronounced similarly to pocket. PokeConv, PokeBNN, and Pokemon are abbreviations of Pocket Convolution, Pocket Binary Neural Network, and Pocket Monster, respectively.2 Source code is available as part of the supplemental materials. The camera-ready version will include the GitHub link and optimize the source for clarity and ease of reproduction.

show abstract

“…We use the bfloat16 ResNet50 v1.5 model as our baseline and implemented our quantized model on top of JAX [4] MLPerf ResNet50 submission. Figure 3 shows the ResNet50 architecture used in this work.…”

Section: Resnet50mentioning

confidence: 99%

“…We implemented a collection of quantization techniques and quantized neural networks on top of the JAX framework [4] and the Flax library [14] to enable fast experimentation, and used it to run the experiments in this paper. All code is at https://github.com/google-research/ google -research / tree / master / aqt.…”

Section: Quantization Library In Jax and Flaxmentioning

confidence: 99%

Pareto-Optimal Quantized ResNet Is Mostly 4-bit

Abdolrashidi¹,

Wang²,

Agrawal³

et al. 2021

Preprint

View full text Add to dashboard Cite

Quantization has become a popular technique to compress neural networks and reduce compute cost, but most prior work focuses on studying quantization without changing the network size. Many real-world applications of neural networks have compute cost and memory budgets, which can be traded off with model quality by changing the number of parameters. In this work, we use ResNet as a case study to systematically investigate the effects of quantization on inference compute cost-quality tradeoff curves. Our results suggest that for each bfloat16 ResNet model, there are quantized models with lower cost and higher accuracy; in other words, the bfloat16 compute cost-quality tradeoff curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve stateof-the-art results on ImageNet for 4-bit ResNet-50 with quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We demonstrate the regularizing effect of quantization by measuring the generalization gap. The quantization method we used is optimized for practicality: It requires little tuning and is designed with hardware capabilities in mind. Our work motivates further research into optimal numeric formats for quantization, as well as the development of machine learning accelerators supporting these formats. As part of this work, we contribute a quantization library written in JAX, which is open-sourced at https : / / github . com / google -research / google-research/tree/master/aqt.

show abstract

Peter Bradbury

Cited by 17 publications

References 0 publications

Baking Neural Radiance Fields for Real-Time View Synthesis

Baking Neural Radiance Fields for Real-Time View Synthesis

PokeBNN: A Binary Pursuit of Lightweight Accuracy

Pareto-Optimal Quantized ResNet Is Mostly 4-bit

Contact Info

Product

Resources

About