Information capacity of the Hopfield model

Abu‐Mostafa, Yaser S.; Jacques, Jeannine‐Marie St.

doi:10.1109/tit.1985.1057069

Cited by 232 publications

(62 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this scheme, the basic ART1 model of Carpenter & Grossberg (1,5) that deals with binary feature vectors is considered as the reference system for developing a workable localized-distributed model for feature recognition/classification. This is outlined as follows.…”

Section: Some Neural Net Classifier Modelsmentioning

confidence: 99%

Advanced Signal Processing Techniques

Andriamanalimanana¹,

Novillo²,

Sengupta³

2004

Signal Processing and Communications

View full text Add to dashboard Cite

Section: Some Neural Net Classifier Modelsmentioning

confidence: 99%

Advanced Signal Processing Techniques

Andriamanalimanana¹,

Novillo²,

Sengupta³

2004

Signal Processing and Communications

View full text Add to dashboard Cite

“…We now need to show that the resulting approximations for the normal entropy and for the binomial entropy are good approximations of each other. (log -> n) 2 /n where C was defined at the beginning of the fi * proof.…”

Section: The Author Believes That the Analysis Would Begin With The Nmentioning

confidence: 99%

Storage Capacity of the Linear Associator: Beginnings of a Theory of Computational Memory

Mumme¹

1988

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

“…These models provide a direction for pattern recognition systems with distinct natural advantages. The capacity of these models, as well as their computing power, are directly related to the number of threshold functions [8].The ability of multilevel threshold devices to simulate a larger number of functions compared to single-threshold devices is vital for the capacity and capabilities of neural network models based on threshold logic. It is therefore of practical as well as theoretical interest to estimate the number of functions that can be modeled as multilevel threshold functions for a given number of inputs and threshold levels.…”

mentioning

confidence: 99%

The capacity of multilevel threshold functions

Olafsson

Abu‐Mostafa

1988

IEEE Trans. Pattern Anal. Machine Intell.

View full text Add to dashboard Cite

Abstract-The investigation of information capacity and computation power of threshold elements is motivated by the recent trend of neural-network approaches to pattern recognition. In this correspondence, we estimate lower and upper bounds for the capacity of multilevel threshold elements, using two essentially different enumeration techniques. The results correct a previously published estimate and indicate that adding threshold levels enhances the capacity more than adding variables.Index Terms-Classifier, information capacity, multilevel threshold element, neural networks, parallel hyperplanes, pattern recognition. I. INTRODUCTIONThreshold functions have somewhat regained interest in recent years. After considerable activity in the early 1960's, the subject all but vanished in the following decade. Current interest is spumed, among other things, by the unexpected difficulty of realizing efficient pattern recognition schemes on conventional digital computers. As an alternative, threshold functions offer a reasonable discrimination capability with simplicity and low cost. When combined in networks, many together create powerful computation networks and storage devices with interesting properties (e.g., [ l ] , [7]). Single threshold devices, the most prevalent type, are found in various forms in nature (most notably the human brain) and can be easily realized physically; that is where the bulk of research has centered [ 11-[4], [ 101. Multithreshold devices, however, have drawn less enthusiasm. Among their qualities, though, is that given enough thresholds, a single multithreshold element can realize any given function operating on a finite domain.There has recently been an intensive interest in threshold logic as the main component of neural network models (e.g., [7]). These models provide a direction for pattern recognition systems with distinct natural advantages. The capacity of these models, as well as their computing power, are directly related to the number of threshold functions [8].The ability of multilevel threshold devices to simulate a larger number of functions compared to single-threshold devices is vital for the capacity and capabilities of neural network models based on threshold logic. It is therefore of practical as well as theoretical interest to estimate the number of functions that can be modeled as multilevel threshold functions for a given number of inputs and threshold levels.In a recent paper [5], the number of dichotomies separable by a multilevel threshold element was derived, unfortunately, incorrectly. Here we will show the claimed result to be a valid lower bound. In addition, we will derive an upper bound using an essentially different argument. We will demonstrate that the exact number of multilevel threshold functions depends strongly on the relative topology of the input set; simple criteria like general position do not fully specify the problem (as opposed to the single-level case).

show abstract

Information capacity of the Hopfield model

Cited by 232 publications

References 5 publications

Advanced Signal Processing Techniques

Advanced Signal Processing Techniques

Storage Capacity of the Linear Associator: Beginnings of a Theory of Computational Memory

The capacity of multilevel threshold functions

Contact Info

Product

Resources

About