Information capacity of a hierarchical neural network

Domínguez, David

doi:10.1103/physreve.58.4811

Cited by 9 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The capability of autoassociative networks to generalize across correlated examples is well-established [7,12,13]. Another line of investigation has also shown that Hopfield-like networks can build a hierarchy of memories that obeys an ultrametric, tree-like structure [14][15][16][17][18][19].…”

Section: Discussionmentioning

confidence: 98%

“…(11)], and m s ≈ γc for dense concepts [Eq. (13)]. These regimes will change as a function the sparse pattern sparsity a, the dense pattern correlation c, the relative dense storage strength γ.…”

Section: A Overview Of Capacity Formulasmentioning

confidence: 99%

“…Under various assumptions, we can simplify the mean field equations Eqs. (10), (11) and (13) to derive formulas for the capacity, or critical example load, of each type of pattern.…”

Section: A Overview Of Capacity Formulasmentioning

confidence: 99%

“…(a) Capacity sc for dense concepts obtained through numerical analysis of Eq. (13). We either set φ = 0 (dark, thin lines) or maximize over φ (light, thick lines).…”

Section: (C1)mentioning

confidence: 99%

See 3 more Smart Citations

A Hopfield-like model with complementary encodings of memories

Kang¹,

Toyoizumi²

2023

Preprint

View full text Add to dashboard Cite

We present a Hopfield-like autoassociative network that stores each memory as two different activity patterns with complementary properties. The first encoding is dense and mutually correlated with a subset of other dense encodings, such that each memory represents an example of a concept. The second encoding is sparse and exhibits no correlation among examples. The network stores each memory as a linear combination of its encodings, which allows for sparse and dense encodings to be retrieved at high and low activity threshold, respectively. At low threshold, as the number of examples stored increases, the retrieved activity shifts from representing densely encoded examples to densely encoded concepts, which are built from accumulating common example features. Meanwhile, at high threshold, the network can still distinctly retrieve many sparsely encoded examples due to the high capacity of sparse, decorrelated patterns. Thus, we demonstrate that a simple autoassociative network with a Hebbian learning rule can retrieve memories at two scales. It can also perform heteroassociation between them, such that one encoding of a memory can be used as a cue to retrieve another. We obtain our results by deriving macroscopic mean-field equations for this network, which allows us to calculate capacity formulas for sparse examples, dense examples, and dense concepts. We also perform network simulations, which verify our theoretical results and explicitly demonstrate the capabilities of our model.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: A Overview Of Capacity Formulasmentioning

confidence: 99%

“…Under various assumptions, we can simplify the mean field equations Eqs. (10), (11) and (13) to derive formulas for the capacity, or critical example load, of each type of pattern.…”

Section: A Overview Of Capacity Formulasmentioning

confidence: 99%

“…(a) Capacity sc for dense concepts obtained through numerical analysis of Eq. (13). We either set φ = 0 (dark, thin lines) or maximize over φ (light, thick lines).…”

Section: (C1)mentioning

confidence: 99%

See 2 more Smart Citations

A Hopfield-like model with complementary encodings of memories

Kang¹,

Toyoizumi²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Our country at present has entered a rapid development period of subway construction, which often takes the form of double-tube parallel tunnel. The double-tube parallel tunnel excavation, because of its cross-impact, will lead to larger superposition of ground surface subsidence, inclination and the cracking destruction [1][2]. This series of unfavorable factors will have an adverse impact on pipeline buried into the surrounding soil, and even cause serious safety hazards on State Property, the normal life of the people [3][4].…”

Section: Introductionmentioning

confidence: 99%

Analysis of Effects of a Double-Tube Parallel Tunnel Excavation on Underground Pipelines

Dong

Zhao

2013

AMM

View full text Add to dashboard Cite

The interaction of a double-tube parallel tunnel excavation will greatly influence on underground pipeline of its region. This paper, taking Dalian Spring Street subway station as the background, uses the finite difference software to establish 3-D finite element analysis model to simulate the effects of tunnel excavation on underground pipeline, in which different embedment, material properties and diameters are considered. By analyzing of the settlement and deformation law of pipeline, the least affected pipeline conditions are drawn to provide corresponding guidance for subsequent underground projects.

show abstract

Latent Attractors: A General Paradigm for Context-Dependent Neural Computation

Doboli

Minai

Studies in Computational Intelligence

View full text Add to dashboard Cite

Information capacity of a hierarchical neural network

Cited by 9 publications

References 12 publications

A Hopfield-like model with complementary encodings of memories

A Hopfield-like model with complementary encodings of memories

Analysis of Effects of a Double-Tube Parallel Tunnel Excavation on Underground Pipelines

Latent Attractors: A General Paradigm for Context-Dependent Neural Computation

Contact Info

Product

Resources

About