Mutual information of sparsely coded associative memory with self-control and ternary neurons

Bollé, Désiré; Domínguez, David; Амари, Шун-ичи

doi:10.1016/s0893-6080(00)00027-7

Cited by 20 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to measure the retrieval quality of the recall process, we use the mutual information function [5,6,13,14]. In general, it measures the average amount of information that can be received by the user by observing the signal at the output of a channel [15,16].…”

Section: The Modelmentioning

confidence: 99%

“…To solve this problem, quite recently a self-control mechanism has been introduced in the dynamics of networks for so-called diluted architectures [5]. This self-control mechanism introduces a timedependent threshold in the transfer function [5,6]. It is determined as a function of both the cross-talk noise and the activity of the stored patterns in the network, and adapts itself in the course of the recall process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Thresholds for Layered Neural Networks with Synaptic Noise

Bollé

Heylen

2006

Artificial Neural Networks – ICANN 2006

View full text Add to dashboard Cite

Abstract. The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of layered feedforward neural network models with synaptic noise. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

show abstract

Section: The Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptive Thresholds for Layered Neural Networks with Synaptic Noise

Bollé

Heylen

2006

Artificial Neural Networks – ICANN 2006

View full text Add to dashboard Cite

show abstract

“…It is known that an important property required for efficient design is an autonomous function independent from, for example, user specified parameters [4]. The use of many user specified parameters requires a user to know rich prior knowledge, which often does not exist for complex real-world problems.…”

Section: B Constructive Approachmentioning

confidence: 99%

A constructive algorithm for training cooperative neural network ensembles

Islam

Yao

Murase

2003

IEEE Trans. Neural Netw.

262

126

View full text Add to dashboard Cite

Abstract-This paper presents a constructive algorithm for training cooperative neural-network ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on both accuracy and diversity among individual NNs in an ensemble. In order to maintain accuracy among individual NNs, the number of hidden nodes in individual NNs are also determined by a constructive approach. Incremental training based on negative correlation is used in CNNE to train individual NNs for different numbers of training epochs. The use of negative correlation learning and different training epochs for training individual NNs reflect CNNEs emphasis on diversity among individual NNs in an ensemble. CNNE has been tested extensively on a number of benchmark problems in machine learning and neural networks, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, soybean, and Mackey-Glass time series prediction problems. The experimental results show that CNNE can produce NN ensembles with good generalization ability.

show abstract

“…It is determined as a function of both the cross-talk noise and the activity of the stored patterns in the network, and adapts itself in the course of the recall process. It furthermore allows to reach optimal retrieval performance both in the absence and in the presence of synaptic noise [5,6,7,8]. These diluted architectures contain no common ancestors nodes, in contrast with feedforward architectures.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Thresholds for Neural Networks With Synaptic Noise

Bollé

Heylen

2007

Int. J. Neur. Syst.

View full text Add to dashboard Cite

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

show abstract

Mutual information of sparsely coded associative memory with self-control and ternary neurons

Cited by 20 publications

References 26 publications

Adaptive Thresholds for Layered Neural Networks with Synaptic Noise

Adaptive Thresholds for Layered Neural Networks with Synaptic Noise

A constructive algorithm for training cooperative neural network ensembles

Adaptive Thresholds for Neural Networks With Synaptic Noise

Contact Info

Product

Resources

About