Binary Optimization: On the Probability of a Local Minimum Detection in Random Search

Kryzhanovsky, Boris; Kryzhanovsky, Vladimir

doi:10.1007/978-3-540-69731-2_10

Cited by 14 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The surprising finding is that this competition results in the enlargement of basins of attraction that lead to especially high total-utility attractors at the expense of low-utility attractors. This is explained in part by the fact that in systems built out of the superposition of many low-order constraints, low-energy (high-utility) attractors necessarily have large basins of attraction [11,38,37,39]. So, the better the attractor the more it is visited, thus the more it is enlarged by learning and the more it is visited in future, and so on.…”

Section: Selfish Changes To Connections and Hebbian Learningmentioning

confidence: 99%

“…3.c) [81,78]. Given that good attractors are large attractors [11,38,37,39], the results above can then be partly explained by the development of an associative memory that simply 'recalls' the best configurations previously visited since these are the configurations that are most frequently visited.…”

Section: Memory Optimisation and Generalisationmentioning

confidence: 99%

“…If it is still true that local equilibria with higher total utility tend to have larger basins of attraction, then those attractors (and generalisations thereof) will be enlarged at the expense of lower-utility attractors. We conjecture that, in systems built out of the superposition of many low-order interactions, this must be true on average [38,37,39]. But of course, this will not be true in every instance -and in systems that have only one equilibrium, as in a single, two-player Prisoner's Dilemma, for example, the enlargement of this attractor will have neither a positive nor a detrimental effect on total utility.…”

Section: Memory Optimisation and Generalisationmentioning

confidence: 99%

See 2 more Smart Citations

Global Adaptation in Networks of Selfish Components: Emergent Associative Memory at the System Scale

2011

View full text Add to dashboard Cite

In some circumstances complex adaptive systems composed of numerous self-interested agents can self-organise into structures that enhance global adaptation, efficiency or function. However, the general conditions for such an outcome are poorly understood and present a fundamental open question for domains as varied as ecology, sociology, economics, organismic biology and technological infrastructure design. In contrast, sufficient conditions for artificial neural networks to form structures that perform collective computational processes such as associative memory/recall, classification, generalisation and optimisation, are well-understood. Such global functions within a single agent or organism are not wholly surprising since the mechanisms (e.g. Hebbian learning) that create these neural organisations may be selected for this purpose, but agents in a multi-agent system have no obvious reason to adhere to such a structuring protocol or produce such global behaviours when acting from individual self-interest. However, Hebbian learning is actually a very simple and fully-distributed habituation or positive feedback principle. Here we show that when self-interested agents can modify how they are affected by other agents (e.g. when they can influence which other agents they interact with) then, in adapting these inter-agent relationships to maximise their own utility, they will necessarily alter them in a manner homologous with Hebbian learning. Multi-agent systems with adaptable relationships will thereby exhibit the same system-level behaviours as neural networks under Hebbian learning. For example, improved global efficiency in multi-agent systems can be explained by the inherent ability of associative memory to generalise by idealising stored patterns and/or creating new combinations of sub-patterns. Thus distributed multi-agent systems can spontaneously exhibit adaptive global behaviours in the same sense, and by the same mechanism, as the organisational principles familiar in connectionist models of organismic learning.Keywords: self-organisation, adaptive networks, Hebbian learning, multi-agent systems, social networks, emergent computation, games on networks. Selfish changes to connections and global adaptationOne of the key open questions in the field of artificial life and adaptive systems is how, if at all, it is possible that a complex system that is not evolved can exhibit adaptation or increased functionality without design. has the potential to play a role in the formation of pre-biotic organisations, or in moving from one level of biological organisation to another [42], for example, but theory to understand exactly what this means or how it might work is limited. In contrast, theory to understand distributed adaptive processes in neural networks is well-developed, but generally assumed to be relevant only to brains and nervous systems 'programmed' to exhibit such adaptation. Here we show that organisational principles familiar in learning neural networks emerge spontaneously in distributed...

show abstract

Section: Selfish Changes To Connections and Hebbian Learningmentioning

confidence: 99%

Section: Memory Optimisation and Generalisationmentioning

confidence: 99%

Section: Memory Optimisation and Generalisationmentioning

confidence: 99%

See 1 more Smart Citation

Global Adaptation in Networks of Selfish Components: Emergent Associative Memory at the System Scale

2011

View full text Add to dashboard Cite

show abstract

“…Specifically, in systems that are built from the superposition of many symmetric pairwise interactions, the depth (with respect to energy) of an attractor basin is positively related to its width (the size of the basin of attraction). A robust relationship between minima depth and basin size14 is complicated by the possibility of correlations between minima15, but minima depth and basin size are, in general, strongly correlated on average as evidenced by recent numerical work16–18. Accordingly, the global minimum is likely to have the biggest basin of attraction.…”

Section: Local Constraint Satisfaction and Associative Memorymentioning

confidence: 99%

Optimization in “self-modeling” complex adaptive systems

2010

View full text Add to dashboard Cite

When a dynamical system with multiple point attractors is released from an arbitrary initial condition, it will relax into a configuration that locally resolves the constraints or opposing forces between interdependent state variables. However, when there are many conflicting interdependencies between variables, finding a configuration that globally optimizes these constraints by this method is unlikely or may take many attempts. Here, we show that a simple distributed mechanism can incrementally alter a dynamical system such that it finds lower energy configurations, more reliably and more quickly. Specifically, when Hebbian learning is applied to the connections of a simple dynamical system undergoing repeated relaxation, the system will develop an associative memory that amplifies a subset of its own attractor states. This modifies the dynamics of the system such that its ability to find configurations that minimize total system energy, and globally resolve conflicts between interdependent variables, is enhanced. Moreover, we show that the system is not merely ' 'recalling' ' low energy states that have been previously visited but ' 'predicting' ' their location by generalizing over local attractor states that have already been visited. This ' 'self-modeling' ' framework, i.e., a system that augments its behavior with an associative memory of its own attractors, helps us better understand the conditions under which a simple locally mediated mechanism of self-organization can promote significantly enhanced global resolution of conflicts between the components of a complex adaptive system. We illustrate this process in random and modular network constraint problems equivalent to graph coloring and distributed task allocation problems.

show abstract

“…(7)) that matches perfectly with the data from Table 1. When 50 L  , the relative error is less than 4 2 10   . The dependence 00 () N   (the second expression of Eq.…”

Section: Methodsmentioning

confidence: 94%

Global Minimum Depth in Edwards-Anderson Model

Karandashev¹,

Kryzhanovsky²

2019

Engineering Applications of Neural Networks

Self Cite

View full text Add to dashboard Cite

In the literature the most frequently cited data are quite contradictory, and there is no consensus on the global minimum value of 2D Edwards-Anderson (2D EA) Ising model. By means of computer simulations, with the help of exact polynomial Schraudolph-Kamenetsky algorithm, we examined the global minimum depth in 2D EA-type models. We found a dependence of the global minimum depth on the dimension of the problem N and obtained its asymptotic value in the limit N→∞. We believe these evaluations can be further used for examining the behavior of 2D Bayesian models often used in machine learning and image processing.

show abstract

Binary Optimization: On the Probability of a Local Minimum Detection in Random Search

Cited by 14 publications

References 16 publications

Global Adaptation in Networks of Selfish Components: Emergent Associative Memory at the System Scale

Global Adaptation in Networks of Selfish Components: Emergent Associative Memory at the System Scale

Optimization in “self-modeling” complex adaptive systems

Global Minimum Depth in Edwards-Anderson Model

Contact Info

Product

Resources

About