Benchmarking Inference Performance of Deep Learning Models on Analog Devices

Fagbohungbe, Omobayode; Qian, Lijun

doi:10.48550/arxiv.2011.11840

Cited by 2 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our realistic PCM model has different noise mean and variance for each input. Using this model is an improvement over prior work that only investigate white Gaussian noise (Upadhyaya et al, 2019;Zhou et al, 2020;Fagbohungbe & Qian, 2020) when storing NN models on noisy storage.…”

Section: Phase Change Memory (Pcm)mentioning

confidence: 99%

See 1 more Smart Citation

Neural Network Compression for Noisy Storage Devices

Berivan¹,

Choi²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices. Although NN model compression has made significant progress, there has been considerably less investigation in the actual physical storage of NN parameters. Conventionally, model compression and physical storage are decoupled, as digital storage media with error correcting codes (ECCs) provide robust error-free storage. This decoupled approach is inefficient, as it forces the storage to treat each bit of the compressed model equally, and to dedicate the same amount of resources to each bit. We propose a radically different approach that: (i) employs analog memories to maximize the capacity of each memory cell, and (ii) jointly optimizes model compression and physical storage to maximize memory utility. We investigate the challenges of analog storage by studying model storage on phase change memory (PCM) arrays and develop a variety of robust coding strategies for NN model storage. We demonstrate the efficacy of our approach on MNIST, CIFAR-10 and ImageNet datasets for both existing and novel compression methods. Compared to conventional error-free digital storage, our method has the potential to reduce the memory size by one order of magnitude, without significantly compromising the stored model's accuracy.

show abstract

Section: Phase Change Memory (Pcm)mentioning

confidence: 99%

“…To the best of our knowledge, this is the first work on NN compression for analog storage devices with realistic noise characteristics, unlike previous work that only investigate white Gaussian noise (Upadhyaya et al, 2019;Zhou et al, 2020;Fagbohungbe & Qian, 2020). In particular, our contributions are:…”

Section: Introductionmentioning

confidence: 99%

Neural Network Compression for Noisy Storage Devices

Berivan¹,

Choi²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We then added AWGN to the well-trained NN weights under a given SNR and used different estimators to denoise the noisy weights. A caveat here is that the parameters of the batch-normalization and the layer-normalization layers were set to be noise-free in the experiments [6,8], because they behave differently than other parameters [20]. These parameters are few in number.…”

Section: A Implementation Detailsmentioning

confidence: 99%

Denoising Noisy Neural Networks: A Bayesian Approach with Compensation

Shao

Liew

Gündüz

2021

Preprint

View full text Add to dashboard Cite

Noisy neural networks (NoisyNNs) refer to the inference and training of NNs in the presence of noise. Noise is inherent in most communication and storage systems; hence, NoisyNNs emerge in many new applications, including federated edge learning, where wireless devices collaboratively train a NN over a noisy wireless channel, or when NNs are implemented/stored in an analog storage medium. This paper studies a fundamental problem of NoisyNNs: how to estimate the uncontaminated NN weights from their noisy observations or manifestations. Whereas all prior works relied on the maximum likelihood (ML) estimation to maximize the likelihood function of the estimated NN weights, this paper demonstrates that the ML estimator is in general suboptimal. To overcome the suboptimality of the conventional ML estimator, we put forth an MMSE pb estimator to minimize a compensated mean squared error (MSE) with a population compensator and a bias compensator. Our approach works well for NoisyNNs arising in both 1) noisy inference, where noise is introduced only in the inference phase on the already-trained NN weights; and 2) noisy training, where noise is introduced over the course of training. Extensive experiments on the CIFAR-10 and SST-2 datasets with different NN architectures verify the significant performance gains of the MMSE pb estimator over the ML estimator when used to denoise the NoisyNN. For noisy inference, the average gains are up to 156% for a noisy ResNet34 model and 14.7% for a noisy BERT model; for noisy training, the average gains are up to 18.1 dB for a noisy ResNet18 model.

show abstract

Benchmarking Inference Performance of Deep Learning Models on Analog Devices

Cited by 2 publications

References 23 publications

Neural Network Compression for Noisy Storage Devices

Neural Network Compression for Noisy Storage Devices

Denoising Noisy Neural Networks: A Bayesian Approach with Compensation

Contact Info

Product

Resources

About