A Convolutional Accelerator for Neural Networks With Binary Weights

Ardakani, Arash; Condo, Carlo; Gross, Warren J.

doi:10.1109/iscas.2018.8350945

Cited by 11 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the fact that the number of zero synapses acts as an indicator of concept formation is intimately related to the spontaneous symmetry breaking in the model. These findings may also have implications on promising deep neuromorphic computation with discrete synapses [26].…”

Section: Discussionmentioning

confidence: 80%

Role of zero synapses in unsupervised feature learning

Huang¹

2018

J. Phys. A: Math. Theor.

View full text Add to dashboard Cite

Synapses in real neural circuits can take discrete values including zero (silent or potential) synapses. The computational role of zero synapses in unsupervised feature learning of unlabeled noisy data is still unclear, thus it is important to understand how the sparseness of synaptic activity is shaped during learning and its relationship with receptive field formation. Here, we formulate this kind of sparse feature learning by a statistical mechanics approach. We find that learning decreases the fraction of zero synapses, and when the fraction decreases rapidly around a critical data size, an intrinsically structured receptive field starts to develop. Further increasing the data size refines the receptive field, while a very small fraction of zero synapses remain to act as contour detectors. This phenomenon is discovered not only in learning a handwritten digits dataset, but also in learning retinal neural activity measured in a natural-movie-stimuli experiment.

show abstract

Section: Discussionmentioning

confidence: 80%

Role of zero synapses in unsupervised feature learning

Huang¹

2018

J. Phys. A: Math. Theor.

View full text Add to dashboard Cite

show abstract

“…The first step consists in performing transfer using the internal layers of a pre-trained DNN acting as a generic feature extractor to compute the feature vector x m , corresponding to the input signal s m . Since there is already a lot of literature on the subject of hardware implementation of the inference of DNNs [11]- [13], we disregard this first step in the implementation described in this paper and directly consider processing the vector x m . Then, each feature vector x m is split into P subvectors of equal size denoted x m p 1≤p≤P .…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Several approaches have been introduced aiming at reducing DNNs size and complexity, such as using product quantization (PQ) to factorize DNNs weights [2], [3], binarizing both DNNs weights and activations [4]- [6], pruning DNN connections [7], [8], or replacing the spatial convolution by a shift operation followed by 1 × 1 convolution [9], [10]. These methods ease the implementation of big and complex DNNs on embedded systems [11]- [13] as far as the inference part is concerned. However, because it requires storing the whole dataset and causes increased complexity, the training of DNNs is in most cases still performed offline.…”

Section: Introductionmentioning

confidence: 99%

Efficient Hardware Implementation of Incremental Learning and Inference on Chip

Hacene

Gripon

Farrugia

et al. 2019

2019 17th IEEE International New Circuits and Systems Conference (NEWCAS)

View full text Add to dashboard Cite

In this paper, we tackle the problem of incrementally learning a classifier, one example at a time, directly on chip. To this end we propose an efficient hardware implementation of a recently introduced incremental learning procedure that achieves state-of-the-art performance by combining transfer learning with majority votes and quantization techniques. The proposed design is able to accommodate for both new examples and new classes directly on the chip. We detail the hardware implementation of the method (implemented on FPGA target) and show it requires limited resources while providing a significant acceleration compared to using a CPU.Index Terms-deep learning, artificial neural networks, incremental, FPGA, hardware.

show abstract

“…A potential manner in which AI can be implemented in IoT devices is via a binarized CNN (BCNN) [3], where network parameters are expressed in a binary format with an inference accuracy comparable to that of the original CNN. Several researches have been conducted on implementing BCNN accelerators using various hardware platforms such as GPUs [4], ASICs [5][6][7][8][9][10][11][12][13][14][15][16], and FPGAs [17][18][19][20][21][22][23][24].…”

Section: Introductionmentioning

confidence: 99%

Design framework for an energy-efficient binary convolutional neural network accelerator based on nonvolatile logic

Suzuki

Oka

Tamakoshi

et al. 2021

NOLTA

View full text Add to dashboard Cite

Convolutional neural network (CNN) accelerators, particularly binarized CNN (BCNN) accelerators have proven to be effective for several artificial-intelligence-oriented several applications; however, their energy efficiency should be further improved for edge applications. In this paper, a design framework for an energy-efficient BCNN accelerator based on nonvolatile logic is presented. Designing BCNN accelerators using nonvolatile logic allows for the accelerators to exhibit a massively parallel and ultra-low standby power capability. Thus, a new design can be realized for accelerators that is different from that of conventional accelerators based solely on CMOS. Considering this, we discuss a concrete design considerations of nonvolatile BCNN accelerators. In fact, a systematic design flow of the nonvolatile BCNN is established by combining Vivado HLS and standard electronic design automation tools. As a typical design example, a BCNN accelerator for inferring 32 × 32 pixel MNIST dataset is designed using a 65-nm CMOS technology. By the logic-synthesis result, the proposed BCNN accelerator is estimated to consume 94.2% lower power than that of a conventional BCNN accelerator when the frame rate is 30 frames per second.

show abstract

A Convolutional Accelerator for Neural Networks With Binary Weights

Cited by 11 publications

References 8 publications

Role of zero synapses in unsupervised feature learning

Role of zero synapses in unsupervised feature learning

Efficient Hardware Implementation of Incremental Learning and Inference on Chip

Design framework for an energy-efficient binary convolutional neural network accelerator based on nonvolatile logic

Contact Info

Product

Resources

About