The accuracy and power consumption of resistive crossbar circuits in use for neuromorphic computing is restricted by the process variation of the resistance-switching (memristive) device and the power overhead of the mixed-signal circuits, such as analog-digital converters (ADCs) and digitalanalog converters (DACs). Reducing the signal-and weightresolution can improve the robustness against process variation, relax requirements for mixed-signal devices, and simplify the implementation of crossbar circuits. This work aims to establish a methodology to achieve low-resolution dense layers for CNNs in terms of network architecture selection and quantization method. To this end, this work studies the impact of the dense layer configuration on the required resolution for its inputs and weights in a small convolutional neural network (CNN). This analysis shows that carefully selecting the network architecture for the dense layer can significantly reduce the required resolution for its input signals and weights. This work reviews criteria for appropriate architecture selection and the quantization method for the binary and ternary neural network (BNN and TNN) to reduce the weight resolution of CNN dense layers. Furthermore, this work presents a method to reduce the input resolution for the dense layer down to one bit by analyzing the distribution of the input values. A small CNN for inference with one-bit quantization for inputs signals and weights can be realized with only 0.68% accuracy degradation for MNIST Dataset.
The size of crossbar-like resistive multiplication accumulation (MAC) accelerators is restricted by parasitic interconnect resistances limiting the computation efficiency. In order to understand the impact of the interconnect (wire) resistance on the computation accuracy, this work proposes a modelling methodology for adapting the neural network weights regarding the weight error caused by wire resistance. Even more, this work investigates the maximal achievable crossbar size based on the proposed design variable "the minimal allowed weight error ratio Ron/Reqv" instead of using an iterative numerical method. Using the proposed method this work investigate the computation accuracy improvement through splitting the layer computation into multiple small crossbars. The result indicates that simply separating the computation into multiple small crossbar does not improve the accuracy always. Considering the minimal Ron/Reqv can more easily result in a proper crossbar size.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.