Stochastic computing in convolutional neural network implementation: a review

Abstract

Stochastic computing (SC) is an alternative computing domain for ubiquitous deterministic computing whereby a single logic gate can perform the arithmetic operation by exploiting the nature of probability math. SC was proposed in the 1960s when binary computing was expensive. However, presently, SC started to regain interest after the widespread of deep learning application, specifically the convolutional neural network (CNN) algorithm due to its practicality in hardware implementation. Although not all computing functions can translate to the SC domain, several useful function blocks related to the CNN algorithm had been proposed and tested by researchers. An evolution of CNN, namely, binarised neural network, had also gained attention in the edge computing due to its compactness and computing efficiency. This study reviews various SC CNN hardware implementation methodologies. Firstly, we review the fundamental concepts of SC and the circuit structure and then compare the advantages and disadvantages amongst different SC methods. Finally, we conclude the overview of SC in CNN and make suggestions for widespread implementation.

Stochastic Computing Convolution Neural Network Architecture Reinvented For Highly Efficient Artificial Intelligence Workload on Field Programmable Gate Array

“…In some cases, binary computing could complement the SC circuit, so mixed domain use cases exist. Review papers published by [ 48 , 49 ] are a good knowledge base for SC before diving deep into the research methodology. Firstly, the SNG was an evident design bottleneck for SC since FPGA has limited resources.…”

SC-IZ: A Low-Cost Biologically Plausible Izhikevich Neuron for Large-Scale Neuromorphic Systems Using Stochastic Computing

“…In each clock cycle, a 1 is generated if the binary number (A) is greater than the number (R) generated by the Linear Feedback Shift Register (LFSR). SC Finite-State Machines (FSMs) have been introduced as methods to perform various non-linear functions, such as exponential functions, Rectified Linear Unit (ReLU) functions, and hyperbolic tangent functions on stochastic bitstreams [46]. Furthermore, a stochastic integrator has been proposed to implement the accumulative function of the Euler method in an ODE solver [40].…”

ANC: Attention Network for COVID-19 Explainable Diagnosis Based on Convolutional Block Attention Module

“…Deep learning (DL) has gained successes in prediction/classification tasks. There are many DL structures, such as deep neural network [29], deep belief network, convolutional neural network (CNN) [30], recurrent neural network [31], graph neural network, etc. Among all those DL structures, CNN is particularly suitable for analyzing visual images.…”