Energy-efficient hybrid stochastic-binary neural networks for near-sensor computing

Abstract

Abstract-Recent advances in neural networks (NNs) exhibit unprecedented success at transforming large, unstructured data streams into compact higher-level semantic information for tasks such as handwriting recognition, image classification, and speech recognition. Ideally, systems would employ near-sensor computation to execute these tasks at sensor endpoints to maximize data reduction and minimize data movement. However, nearsensor computing presents its own set of challenges such as operating power constraints, energy budgets, and communication bandwidth capacities. In this paper, we propose a stochasticbinary hybrid design which splits the computation between the stochastic and binary domains for near-sensor NN applications. In addition, our design uses a new stochastic adder and multiplier that are significantly more accurate than existing adders and multipliers. We also show that retraining the binary portion of the NN computation can compensate for precision losses introduced by shorter stochastic bit-streams, allowing faster run times at minimal accuracy losses. Our evaluation shows that our hybrid stochastic-binary design can achieve 9.8× energy efficiency savings, and application-level accuracies within 0.05% compared to conventional all-binary designs.

Removing constant‐induced errors in stochastic circuits

“…4 b , implying that this T‐flip‐flop‐based design is also an OMC circuit. This confirms the high accuracy claimed for the T‐flip‐flop‐based adder, a key factor in the success of the neural network implementation in [15].…”

“…A scaled sequential adder constructed in ad hoc fashion around a T flip‐flop is given in [15] and shown by simulation to be more accurate than the standard combinational design. The STG of that adder is exactly the same as the CEASE‐generated one shown in Fig.…”

A Cognitive Stochastic Machine Based on Bayesian Inference: A Behavioral Analysis

“…Working on stochastic hardware, a stochastic adder has been presented in [17] which gives new opportunities to develop stochastic hardware for Bayesian inference. The robustness of Bayesian machines has been demonstrated in [4] by fault injections.…”

High Quality Down-Sampling for Deterministic Approaches to Stochastic Computing

“…Logical computation is performed on probability data represented by uniformly distributed random bit-streams. Image and video processing [2], [5], [14], [20], digital filters [15], low-density parity check decoding [16], [22] and neural networks [3], [4], [9], [11], [12], [13] have been the main target applications for SC. Low hardware cost and low power consumption advantages of this computing paradigm have encouraged designers to implement complex calculations in the stochastic domain.…”