Probabilistic Interpretation of Feedforward Network Outputs, With Relationships to Statistical Prediction of Ordinal Quantities

Abstract

Several problems require the estimation of discrete random variables whose values can be put in a one-to-one ordered correspondence with a finite subset of the natural numbers. This happens whenever quantities are involved that represent integer items, or have been quantized on a fixed number of levels, or correspond to “graded” linguistic values. Here we propose a correct probabilistic approach to such kind of problems that fully exploits all the available prior knowledge about their own structure. In spite of the very stringent constraints induced in output space, the method can be directly applied to standard feed-forward networks while keeping local computation of both outputs and error signals. According to these guidelines, we devised a neural implementation of a complex image pre-processing algorithm by using very poor resolution on the computing elements in the network.

Ordinal Regression Methods: Survey and Experimental Study

“…For hyperbolic functions, negative symbols are represented with a −1 and positive ones also with a 1. Those decompositions where a class is not involved should be treated as a "does not matter" condition where, whatever the output response, no error signal should be generated [69].…”

“…A different approach is taken in [74], where the ordinal constraints are included into the weights connecting the hidden and output layers. Costa [69] followed a probabilistic framework to propose another neural network architecture able to exploit the ordinal nature of the data. The proposal is based on the joint prediction of constrained concurrent events, which can be turned into a classification task defined in a suitable space through a "partitive approach".…”

Classification of Ordinal Data Using Neural Networks

“…Finally, for 10 classes the rank was assigned according to the rule y = m i n r∈{1, 2,3,4,5,6,7,8,9,10} {r : br−1 < 1000 b1, b2, b3, b4, b5, b6, b7, b8, where ε is a random value, normally distributed with zero mean and σ = 0.125. The results are presented in a tabular form, tables 1 and 2.…”

“…To predict the class value of an unseen instance the probabilities of the K original classes are estimated using the outputs from the K − 1 binary classifiers. -Costa [5], following a probabilistic approach, proposes a neural network architecture (itNN) that exploits the ordinal nature of the data, by defining the classification task on a suitable space through a "partitive approach". It is proposed a feedforward neural network with K − 1 outputs to solve a K-class ordinal problem.…”

Ordinal Neural Networks Without Iterative Tuning

“…On the other hand, Costa [33] proposes a probabilistic neural network for the ordinal scenario. To adapt neural networks to the ordinal case structure, targets are reformulated following the OneVsFollowers approach and the prediction phase is realized considering that the output of the j th output neuron is estimating the probability that j and j − 1 ranks are both truth [28], [34], [35].…”