Training binary node feedforward neural networks by back propagation of error

Toms, D.J.

doi:10.1049/el:19901121

Cited by 32 publications

(19 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent research publications have tried to alleviate this problem by considering various modifications of the gradient descent, such as the MRII algorithm [28]. Another training method was proposed by Toms [25], who suggested the use of hybrid activations that are gradually transformed during training from analogue (sigmoid) to thresholds (step functions) depending on the values of a heuristic parameter b, 0 £ b £ 1. Thus, the hidden unit activations, r h , are…”

Section: Training Network With Threshold Activation Functionsmentioning

confidence: 99%

See 1 more Smart Citation

Evolutionary training of hardware realizable multilayer perceptrons

Plagianakos

Magoulas

Vrahatis

2005

Neural Comput & Applic

View full text Add to dashboard Cite

The use of multilayer perceptrons (MLP) with threshold functions (binary step function activations) greatly reduces the complexity of the hardware implementation of neural networks, provides tolerance to noise and improves the interpretation of the internal representations. In certain case, such as in learning stationary tasks, it may be sufficient to find appropriate weights for an MLP with threshold activation functions by software simulation and, then, transfer the weight values to the hardware implementation. Efficient training of these networks is a subject of considerable ongoing research. Methods available in the literature mainly focus on two-state (threshold) nodes and try to train the networks by approximating the gradient of the error function and modifying appropriately the gradient descent, or by progressively altering the shape of the activation functions. In this paper, we propose an evolution-motivated approach, which is eminently suitable for networks with threshold functions and compare its performance with four other methods. The proposed evolutionary strategy does not need gradient related information, it is applicable to a situation where threshold activations are used from the beginning of the training, as in ''on-chip'' training, and is able to train networks with integer weights.

show abstract

Section: Training Network With Threshold Activation Functionsmentioning

confidence: 99%

“…Various modifications of the gradient descent have been presented to train MLPs with threshold activations [2,5,6,12,25,28]. However, these methods require to a certain degree, depending on the case, that the learning task is static.…”

Section: Introductionmentioning

confidence: 99%

Evolutionary training of hardware realizable multilayer perceptrons

Plagianakos

Magoulas

Vrahatis

2005

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…In this section, we present comparative results for the DE algorithm, and the algorithms proposed in [4], [14], [3], [6], which are denoted in the tables below as (GLO), (T), (GZ), and (MVGA), respectively.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Another method proposed by Tom [14] uses a hybrid activation function that is a linear combination of analog (sigmoid) and discrete (hard-limiting) functions depending on the values of a heuristic parameter. Thus, for a specific range of values a unit can be purely analog having a sigmoid activation, such as in Relation (3), while for other values it becomes purely binary.…”

Section: Training Methods For Network With Discrete Activationsmentioning

confidence: 99%

See 1 more Smart Citation

Training multilayer networks with discrete activation functions

Plagianakos

Magoulas

Nousis

et al.

IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222)

View full text Add to dashboard Cite

Efficient training of multilayer networks with discrete activation functions is a subject of considerable ongoing research. The use of these networks greatly reduces the complexity of the hardware implementation, provides tolerance to noise and improves the interpretation of the internal representations. Methods available in the literature mainly focus on two-state (binary) nodes and try to train these networks by approximating the gradient and modifying appropriately the gradient descent. However, they exhibit slow convergence speed and low possibility of success compared to networks with continuous activations. In this work, we propose an evolution-motivated approach, which is eminently suitable for networks with discrete output states and compare its performance with four other methods.

show abstract

Benchmarking Differential Evolution

Natural Computing Series

View full text Add to dashboard Cite

Training binary node feedforward neural networks by back propagation of error

Cited by 32 publications

References 1 publication

Evolutionary training of hardware realizable multilayer perceptrons

Evolutionary training of hardware realizable multilayer perceptrons

Training multilayer networks with discrete activation functions

Benchmarking Differential Evolution

Contact Info

Product

Resources

About