Handwritten digit recognition: applications of neural network chips and automatic learning

Cun, Y. Le; Jackel, L. D.; Boser, Bernhard E.; Denker, John S.; Graf, Hans Peter; Guyon, Isabelle; Henderson, D.; Howard, Richard; Hubbard, W.

doi:10.1109/35.41400

Cited by 404 publications

(190 citation statements)

References 5 publications

Supporting

Mentioning

164

Contrasting

Unclassified

Order By: Relevance

“…For instance, a simple run length coding that involves signaling non-zero values of 16-bits and then runs of zeros up to 31 can reduce the external memory bandwidth of the activations by 2.1× and the overall external bandwidth (including weights) by 1.5× [61]. 11 In addition to compression, the hardware can also be modified such that it skips reading the weights and performing the MAC for zero-valued activations to reduce energy cost by 45% [94]. Rather than just gating the read and MAC computation, the hardware could also skip the cycle to increase the throughput by 1.37× [138].…”

Section: B Reduce Number Of Operations and Model Sizementioning

confidence: 99%

“…Although neural nets were proposed in the 1940s, the first practical application employing multiple digital neurons didn't appear until the late 1980s with the LeNet network for handwritten digit recognition [11] 3 . Such systems are widely used by ATMs for digit recognition on checks.…”

Section: Development Historymentioning

confidence: 99%

“…LeNet [11] was one of the first CNN approaches introduced in 1989. It was designed for the task of digit classification in grayscale images of size 28×28.…”

Section: B Popular Dnn Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

et al. 2017

View full text Add to dashboard Cite

Abstract-Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems.This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry.The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

show abstract

Section: B Reduce Number Of Operations and Model Sizementioning

confidence: 99%

Section: Development Historymentioning

confidence: 99%

See 1 more Smart Citation

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

et al. 2017

View full text Add to dashboard Cite

show abstract

“…These domains include military applications [20,21,9], human face recognition [3,22], agricultural product classification [23], handwritten character recognition [24,25], medical image analysis [26], postal code recognition [27,28], and texture classification [29].…”

Section: Related Workmentioning

confidence: 99%

Using Gaussian distribution to construct fitness functions in genetic programming for multiclass object classification

Zhang

Smart

2006

Pattern Recognition Letters

View full text Add to dashboard Cite

This paper describes a new approach to the use of Gaussian distribution in genetic programming (GP) for multiclass object classification problems. Instead of using predefined multiple thresholds to form different regions in the program output space for different classes, this approach uses probabilities of different classes, derived from Gaussian distributions, to construct the fitness function for classification. Two fitness measures, overlap area and weighted distribution distance, have been developed. Rather than using the best evolved program in a population, this approach uses multiple programs and a voting strategy to perform classification. The approach is examined on three multiclass object classification problems of increasing difficulty and compared with a basic GP approach. The results suggest that the new approach is more effective and more efficient than the basic GP approach. Although developed for object classification, this approach is expected to be able to be applied to other classification problems.Keywords Probability based genetic programming, object recognition, object detection, fitness function, multiclass classification Author InformationThe first author is a staff member in computer science, Victoria University of Wellington, New Zealand. The second author is postgraduate student in computer science, Victoria University of Wellington, New Zealand. Using Gaussian Distribution to Construct Fitness Functions in Genetic Programming AbstractThis paper describes a new approach to the use of Gaussian distribution in genetic programming (GP) for multiclass object classification problems. Instead of using predefined multiple thresholds to form different regions in the program output space for different classes, this approach uses probabilities of different classes, derived from Gaussian distributions, to construct the fitness function for classification. Two fitness measures, overlap area and weighted distribution distance, have been developed. Rather than using the best evolved program in a population, this approach uses multiple programs and a voting strategy to perform classification. The approach is examined on three multiclass object classification problems of increasing difficulty and compared with a basic GP approach. The results suggest that the new approach is more effective and more efficient than the basic GP approach. Although developed for object classification, this approach is expected to be able to be applied to other classification problems.

show abstract

“…Convolutional neural networks have been used for image classification since pioneering research by Yann LeCun in 1980ies [3]. More recently, since the work of A. Krizhevsky [4] the same basic concepts are applied to classification of colorful images (photos).…”

mentioning

confidence: 99%

How effective is Transfer Learning method for image classification

Dabrowski¹,

Michalik²

2017

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Abstract-This paper deals with re-training neural networkbased image classification model, using so-called Transfer Learning approach. This method allows for creating a new image classifier, reusing pre-trained weights from a publicly available model. Our study gives some insight on accuracy of re-trained models and provides guidelines concerning required number of training examples. Presented results may be useful for computer vision practitioners, who would like to adapt results of state-of-the-art research on neural networks for their own customized image recognition models.

show abstract

Handwritten digit recognition: applications of neural network chips and automatic learning

Cited by 404 publications

References 5 publications

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Using Gaussian distribution to construct fitness functions in genetic programming for multiclass object classification

How effective is Transfer Learning method for image classification

Contact Info

Product

Resources

About