Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

Henwood, Sébastien; Leduc-Primeau, François; Savaria, Yvon

doi:10.1109/aicas48895.2020.9073854

Cited by 5 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training with errors may not be desired because the parameters are modified, while injecting errors during training is very time consuming or may even be infeasible in some scenarios. Furthermore, using the same p for every layer in the BNNs may not be an optimal option for p, since different layers in NNs exhibit different sensitivity to error [26], [27].…”

Section: B Error-resilient Bnns During Run-timementioning

confidence: 99%

Reliable Binarized Neural Networks on Unreliable Beyond Von-Neumann Architecture

Yayla

Thomann

Buschjäger

et al. 2022

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

Specialized hardware accelerators beyond von-Neumann, that offer processing capability in where the data resides without moving it, become inevitable in data-centric computing. Emerging non-volatile memories, like Ferroelectric Field-Effect Transistor (FeFET), are able to build compact Logic-in-Memory (LiM). In this work, we investigate the probability of error (Perror) in FeFET-based XNOR LiM, demonstrating the new trade-off between the speed and reliability. Using our reliability model, we present how Binarized Neural Networks (BNNs) can be proactively trained in the presence of XNOR-induced errors towards obtaining robust BNNs at the design time. Furthermore, leveraging the trade-off between Perror and speed, we present a run-time adaptation technique, that selectively trades-off Perror and XNOR speed for every BNN layer. Our results demonstrate that when a small loss (e.g., 1%) in inference accuracy could be accepted, our design-time and run-time techniques provide error-resilient BNNs that exhibit 75% and 50% (FashionMNIST) and 38% and 24% (CIFAR10) XNOR speedups, respectively.

show abstract

Section: B Error-resilient Bnns During Run-timementioning

confidence: 99%

Reliable Binarized Neural Networks on Unreliable Beyond Von-Neumann Architecture

Yayla

Thomann

Buschjäger

et al. 2022

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

show abstract

“…voltage is scaled in [38], [39]. Yang et al [40] seperately tune weight and activation values of BNNs to achieve finegrained control over energy comsumption.…”

Section: Related Workmentioning

confidence: 99%

FeFET-Based Binarized Neural Networks Under Temperature-Dependent Bit Errors

Yayla

Buschjäger

Gupta

et al. 2022

IEEE Trans. Comput.

View full text Add to dashboard Cite

Ferroelectric FET (FeFET) is a highly promising emerging non-volatile memory (NVM) technology, especially for binarized neural network (BNN) inference on the low-power edge. The reliability of such devices, however, inherently depends on temperature. Hence, changes in temperature during run time manifest themselves as changes in bit error rates. In this work, we reveal the temperature-dependent bit error model of FeFET memories, evaluate its effect on BNN accuracy, and propose countermeasures. We begin on the transistor level and accurately model the impact of temperature on bit error rates of FeFET. This analysis reveals temperature-dependent asymmetric bit error rates. Afterwards, on the application level, we evaluate the impact of the temperature-dependent bit errors on the accuracy of BNNs. Under such bit errors, the BNN accuracy drops to unacceptable levels when no countermeasures are employed. We propose two countermeasures: (1) Training BNNs for bit error tolerance by injecting bit flips into the BNN data, and (2) applying a bit error rate assignment algorithm (BERA) which operates in a layer-wise manner and does not inject bit flips during training. In experiments, the BNNs, to which the countermeasures are applied to, effectively tolerate temperature-dependent bit errors for the entire range of operating temperature.

show abstract

“…In order to reduce its energy consumption, the quantized Kalman filter can be implemented on unreliable hardware [ 8 , 10 , 11 , 12 ]. Here, we assume, as in [ 10 , 12 ], that only the memory is faulty. In this case, each memory cell of a memory bank has a bit flipping probability p .…”

Section: System Modelmentioning

confidence: 99%

“…The robustness to unreliability in computation operations and memories has been investigated for several signal processing and machine-learning applications, including binary recursive estimation [ 10 ], binary linear transformation [ 11 ], deep neural networks [ 12 , 13 ], multi-agent systems [ 14 ] and distributed logistic regression [ 15 ]. Moreover, several techniques have been proposed to compensate for faults introduced by unreliable systems.…”

Section: Introductionmentioning

confidence: 99%

Optimizing the Energy Efficiency of Unreliable Memories for Quantized Kalman Filtering

Kern

Dupraz

Bey

et al. 2022

Sensors

View full text Add to dashboard Cite

This paper presents a quantized Kalman filter implemented using unreliable memories. We consider that both the quantization and the unreliable memories introduce errors in the computations, and we develop an error propagation model that takes into account these two sources of errors. In addition to providing updated Kalman filter equations, the proposed error model accurately predicts the covariance of the estimation error and gives a relation between the performance of the filter and its energy consumption, depending on the noise level in the memories. Then, since memories are responsible for a large part of the energy consumption of embedded systems, optimization methods are introduced to minimize the memory energy consumption under the desired estimation performance of the filter. The first method computes the optimal energy levels allocated to each memory bank individually, and the second one optimizes the energy allocation per groups of memory banks. Simulations show a close match between the theoretical analysis and experimental results. Furthermore, they demonstrate an important reduction in energy consumption of more than 50%.

show abstract

Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

Cited by 5 publications

References 15 publications

Reliable Binarized Neural Networks on Unreliable Beyond Von-Neumann Architecture

Reliable Binarized Neural Networks on Unreliable Beyond Von-Neumann Architecture

FeFET-Based Binarized Neural Networks Under Temperature-Dependent Bit Errors

Optimizing the Energy Efficiency of Unreliable Memories for Quantized Kalman Filtering

Contact Info

Product

Resources

About