Maximum likelihood training of probabilistic neural networks

Abstract

A maximum likelihood method is presented for training probabilistic neural networks (PNN's) using a Gaussian kernel, or Parzen window. The proposed training algorithm enables general nonlinear discrimination and is a generalization of Fisher's method for linear discrimination. Important features of maximum likelihood training for PNN's are: 1) it economizes the well known Parzen window estimator while preserving feedforward NN architecture, 2) it utilizes class pooling to generalize classes represented by small training sets, 3) it gives smooth discriminant boundaries that often are "piece-wise flat" for statistical robustness, 4) it is very fast computationally compared to backpropagation, and 5) it is numerically stable. The effectiveness of the proposed maximum likelihood training algorithm is assessed using nonparametric statistical methods to define tolerance intervals on PNN classification performance.

On A Maximum Likelihood Method for Clustering

“…For examples, we can think of its realization by the probabilistic network of Streit and Luginbuhl (1994), and by one of the winner take-all networks.…”

“…The mixture distribu tion is generally assumed in a classification problem, and its unknown parameters can be estimated using a maximum likelihood method (Streit and Luginbuhl, 1994). Of course, while the estimation in a clustering problem may be carried out in much the same way as that in a classification problem, the absence of a priori class memberships is the major difference between these two problem, and it makes the clustering problem much more difficult ; that is, the estimation will be possibly disturbed by the problem of local minima.…”

Assessment of prediction ability for reduced probabilistic neural network in data classification problems

“…It is achieved by estimating probability density functions as a mixture of Gaussian densities with varying covariance matrices. In the reference Streit and Luginbuhl (1994), a maximum likelihood algorithm for training the network is presented as the generalization of Fisher method for nonlinear discrimination. It is shown that the proposed PNN requires significantly fewer nodes and interconnection weights than the original model.…”

Critical Aspects of Supervised Pattern Recognition Methods for Interpreting Compositional Data

“…Kohonen Class-Modelling (KCM) (Marini et al, 2005), and Probabilistic Neural Networks (PNN) (Streit & Luginbuhl, 1994 (Vapnik, 1998;Abe, 2005;Burges, 1998). The purpose of SVM is separate the classes in a vectorial space independently on the probabilistic distribution of pattern vectors in the data set (Berrueta et al, 2007).…”