Learning vector quantization, multi layer perceptron and dynamic programming: comparison and cooperation

Driancourt, X.; Bottou, Léon; Gallinari, Patrick

doi:10.1109/ijcnn.1991.155439

Cited by 16 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After the publication of numerous studies about evaluation and comparison of classical or neural classifiers (Lang et al 1990, Martin&Pittman 1991, Lee 1991 there is a recent trend attempting to design cooperative schemes that would combine the advantages of different approaches (Bollivier et al 1990, Bottou et al 1990, Driancourt et al 1991. These cooperation schemes have been essentially experimented on speech recognition problems.…”

Section: Introductionmentioning

confidence: 97%

<title>Pattern recognition by cooperating neural networks</title>

Idan

Auger²

1992

SPIE Proceedings

View full text Add to dashboard Cite

The study of connectionist models for pattern recognition is mainly motivated by their presumed simultaneous feature selection and classification. Character recognition is a common test case to illustrate the feature extraction and classification characteristics of neural networks. Most of the variability concerning size and rotation can be handled easily, while acquisition conditions are usually controlled. Many examples of neural character recognition applications were presented where the most successful results for optical character recognition (OCR) with image inputs were reported on a layered network (LeCun et al. 1990) integrating feature selection and invariance notions introduced earlier in Neocognitron networks. Previously, we have presented a supervised learning algorithm, based on Kohonen's self-organizing feature maps, and its applications to image and speech processing . From pattern recognition point of view, the first network performs local feature extraction while the second does a global statistical template matching. We describe these models and their comparative results when applied to a common French handwritten zip-code database. We discuss possible cooperation schemes and show that the performances obtained by these networks working in parallel exceed those of the networks working separately. We conclude by the possible extensions of this work for automatic document processing systems.

show abstract

Section: Introductionmentioning

confidence: 97%

<title>Pattern recognition by cooperating neural networks</title>

Idan

Auger²

1992

SPIE Proceedings

View full text Add to dashboard Cite

show abstract

“…The second category is energy-based [14]- [16], which were mainly contributed by B. Juang and Y. LeCun et al, and summarized in [3]. For these loss functions, the corresponding models are viewed as an energy function which measures the "goodness" of each possible configuration of input data and labels.…”

Section: Introductionmentioning

confidence: 99%

Approximating the Gradient of Cross-Entropy Loss Function

Doroslovački

Loew

2020

IEEE Access

View full text Add to dashboard Cite

A loss function has two crucial roles in training a conventional discriminant deep neural network (DNN): (i) it measures the goodness of classification and (ii) generates the gradients that drive the training of the network. In this paper, we approximate the gradients of cross-entropy loss which is the most often used loss function in the classification DNNs. The proposed approximations are noise-free, which means they depend only on the labels of the training set. They have a fixed length to avoid the vanishing gradient problem of the cross-entropy loss. By skipping the forward pass, the computational complexities of the proposed approximations are reduced to O(n) where n is the batch size. Two claims are established based on the experiments of training DNNs using the proposed approximations: (i) It is possible to train a discriminant network without explicitly defining a loss function. (ii) The success of training does not imply the convergence of network parameters to fixed values. The experiments show that the proposed gradient approximations achieve comparable classification accuracy to the conventional loss functions and can accelerate the training process on multiple datasets.

show abstract

A New Unit Selection Optimisation Algorithm for Corpus-Based TTS Systems Using the RBF-Based Data Compression Technique

Rojc

Mlakar

2019

IEEE Access

View full text Add to dashboard Cite

Learning vector quantization, multi layer perceptron and dynamic programming: comparison and cooperation

Cited by 16 publications

References 3 publications

<title>Pattern recognition by cooperating neural networks</title>

<title>Pattern recognition by cooperating neural networks</title>

Approximating the Gradient of Cross-Entropy Loss Function

A New Unit Selection Optimisation Algorithm for Corpus-Based TTS Systems Using the RBF-Based Data Compression Technique

Contact Info

Product

Resources

About