Object Recognition with Gradient-Based Learning

LeCun, Yann; Haffner, Patrick; Bottou, Léon; Bengio, Yoshua

doi:10.1007/3-540-46805-6_19

Cited by 741 publications

(394 citation statements)

References 18 publications

Supporting

Mentioning

352

Contrasting

Unclassified

Order By: Relevance

“…On the learning side, methods for classifying the feature space have ranged from simple nearest neighbor schemes to more complex approaches such as neural networks [8], convolutional neural networks [17], probabilistic methods [11], [18] and linear or higher degree polynomial classifiers [13], [16].…”

Section: A Related Workmentioning

confidence: 99%

Learning to detect objects in images via a sparse, part-based representation

Agarwal

Awan

Roth

2004

IEEE Trans. Pattern Anal. Machine Intell.

734

652

View full text Add to dashboard Cite

Abstract-We study the problem of detecting objects in still, grayscale images. Our primary focus is development of a learning-based approach to the problem, that makes use of a sparse, part-based representation. A vocabulary of distinctive object parts is automatically constructed from a set of sample images of the object class of interest; images are then represented using parts from this vocabulary, together with spatial relations observed among the parts. Based on this representation, a learning algorithm is used to automatically learn to detect instances of the object class in new images. The approach can be applied to any object with distinguishable parts in a relatively fixed spatial configuration; it is evaluated here on difficult sets of real-world images containing side views of cars, and is seen to successfully detect objects in varying conditions amidst background clutter and mild occlusion. In evaluating object detection approaches, several important methodological issues arise that have not been satisfactorily addressed in previous work. A secondary focus of this paper is to highlight these issues and to develop rigorous evaluation standards for the object detection problem. A critical evaluation of our approach under the proposed standards is presented.

show abstract

Section: A Related Workmentioning

confidence: 99%

Learning to detect objects in images via a sparse, part-based representation

Agarwal

Awan

Roth

2004

IEEE Trans. Pattern Anal. Machine Intell.

734

652

View full text Add to dashboard Cite

show abstract

“…Our experiments used the LeNet5 style architecture (LeCun et al 1999), which has already been successfully used for OCR. The main difference between our net, shown in Figure 1, and LeNet5 is that our last layer is a combination of LeNet5's F6 and Output layer.…”

Section: Methodsmentioning

confidence: 99%

“…This method is employed by Deep Convolutional Neural Nets (DCNN's) (LeCun et al 1999). Although these nets are usually referred to as just Convolutional Neural Nets, we refer to them as Deep Convolutional Neural Nets, in order to equally emphasize the use of a deep architecture as well as locally receptive fields.…”

Section: Background and Related Workmentioning

confidence: 99%

Knowledge Transfer in Deep Convolutional Neural Nets

Gutstein

Fuentes

Freudenthal

2008

Int. J. Artif. Intell. Tools

View full text Add to dashboard Cite

Knowledge transfer is widely held to be a primary mechanism that enables humans to quickly learn new complex concepts when given only small training sets. In this paper, we apply knowledge transfer to deep convolutional neural nets, which we argue are particularly well suited for knowledge transfer. Our initial results demonstrate that components of a trained deep convolutional neural net can constructively transfer information to another such net. Furthermore, this transfer is completed in such a way that one can envision creating a net that could learn new concepts throughout its lifetime.

show abstract

“…In the case of SVMs with linear kernel, they showed that for a very large scale problem, widely used SVM packages, including LIBSVM (Chang and Lin 2001) and SVMlight (Joachims 1998(Joachims , 2006, took hours to converge, but SGD only took less than 10 seconds. In fact, about ten years ago, LeCun et al (1998b) and LeCun et al (1999) already showed that SGD is feasible for training multiple-layered hierarchical models for large scale hand-written digit recognition and object recognition tasks. Their results lead the machine learning community re-consider SGD as a useful optimization algorithm and sheds new light on large scale structured prediction.…”

Section: Related Workmentioning

confidence: 99%

Periodic step-size adaptation in second-order gradient descent for single-pass on-line structured learning

et al. 2009

View full text Add to dashboard Cite

It has been established that the second-order stochastic gradient descent (SGD) method can potentially achieve generalization performance as well as empirical optimum in a single pass through the training examples. However, second-order SGD requires computing the inverse of the Hessian matrix of the loss function, which is prohibitively expensive for structured prediction problems that usually involve a very high dimensional feature space. This paper presents a new second-order SGD method, called Periodic Step-size Adaptation (PSA). PSA approximates the Jacobian matrix of the mapping function and explores a linear relation between the Jacobian and Hessian to approximate the Hessian, which is proved to be simpler and more effective than directly approximating Hessian in an on-line setting. We tested PSA on a wide variety of models and tasks, including large scale sequence labeling tasks using conditional random fields and large scale classification tasks using linear support vector machines and convolutional neural networks. Experimental results show that single-pass performance of PSA is always very close to empirical optimum.

show abstract

Object Recognition with Gradient-Based Learning

Cited by 741 publications

References 18 publications

Learning to detect objects in images via a sparse, part-based representation

Learning to detect objects in images via a sparse, part-based representation

Knowledge Transfer in Deep Convolutional Neural Nets

Periodic step-size adaptation in second-order gradient descent for single-pass on-line structured learning

Contact Info

Product

Resources

About