On the Probability of Finding Local Minima in Optimization Problems

Kryzhanovsky, Boris; Magomedov, B. M.; Fonarev, A.

doi:10.1109/ijcnn.2006.247318

Cited by 18 publications

(14 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, in systems that are built from the superposition of many symmetric pairwise interactions, the depth (with respect to energy) of an attractor basin is positively related to its width (the size of the basin of attraction). A robust relationship between minima depth and basin size14 is complicated by the possibility of correlations between minima15, but minima depth and basin size are, in general, strongly correlated on average as evidenced by recent numerical work16–18. Accordingly, the global minimum is likely to have the biggest basin of attraction.…”

Section: Local Constraint Satisfaction and Associative Memorymentioning

confidence: 99%

Optimization in “self-modeling” complex adaptive systems

2010

View full text Add to dashboard Cite

When a dynamical system with multiple point attractors is released from an arbitrary initial condition, it will relax into a configuration that locally resolves the constraints or opposing forces between interdependent state variables. However, when there are many conflicting interdependencies between variables, finding a configuration that globally optimizes these constraints by this method is unlikely or may take many attempts. Here, we show that a simple distributed mechanism can incrementally alter a dynamical system such that it finds lower energy configurations, more reliably and more quickly. Specifically, when Hebbian learning is applied to the connections of a simple dynamical system undergoing repeated relaxation, the system will develop an associative memory that amplifies a subset of its own attractor states. This modifies the dynamics of the system such that its ability to find configurations that minimize total system energy, and globally resolve conflicts between interdependent variables, is enhanced. Moreover, we show that the system is not merely ' 'recalling' ' low energy states that have been previously visited but ' 'predicting' ' their location by generalizing over local attractor states that have already been visited. This ' 'self-modeling' ' framework, i.e., a system that augments its behavior with an associative memory of its own attractors, helps us better understand the conditions under which a simple locally mediated mechanism of self-organization can promote significantly enhanced global resolution of conflicts between the components of a complex adaptive system. We illustrate this process in random and modular network constraint problems equivalent to graph coloring and distributed task allocation problems.

show abstract

Section: Local Constraint Satisfaction and Associative Memorymentioning

confidence: 99%

Optimization in “self-modeling” complex adaptive systems

2010

View full text Add to dashboard Cite

show abstract

“…In Appendix A we show that product is a normallỹ h S s distributed quantity whose mean is and variance Correspondingly, we get for the sought for probability where (4) and is the depth of minimum S 0 . Then the probability of convergence from the n vicinity to minimum S 0 takes the form: (5) By substituting this expression in (3), using the Stirling relation and passing from summation to integra tion with respect to x = n/N, we get for the volume of the attraction area: (6) where (7) We use the saddle point method to evaluate integral (6).…”

Section: The Attraction Area Of a Minimummentioning

confidence: 87%

“…Fortunately, when a neural net is initialized at random, it is most likely to converge to a state corresponding to the global minimum [4,5]. The probability of the net con verging to a minimum not as deep as the global one is slightly lower, while the probability of convergence to small depth local minima is exponentially small.…”

Section: Introductionmentioning

confidence: 99%

Attraction area of minima in quadratic binary optimization

Karandashev

Kryzhanovsky

2014

Opt. Mem. Neural Networks

Self Cite

View full text Add to dashboard Cite

The paper deals with the problem of quadratic functional minimization in the space of binary variables. We analyze the efficiency of the random search procedure used in binary minimiza tion and show that the radius of the attraction area of a minimum directly depends on its depth: the deeper the minimum, the greater its radius of attraction. Thus, the probability of finding a minimum during random search grows exponentially with depth of the minimum.

show abstract

“…However, it is not necessary for a heuristic algorithm to locate the best solution but a local minimum is considered good as long as the requirement is met. The Cost function may manifest several minima in the solution space [10] and can be defined as Cost(Sl) Cost(Sm), Sm N(S) and m M (1) where Sm is a solution produced by a small perturbation to solution S and the perturbation can be called a move, and M is a collection of valid moves that can be made to S. N(S) is the "neighbourhood" of S which is a set of solutions based upon moves in M. Sl is a local minimum if the cost of Sl is lower than any solution in N(S). One of the core algorithms of ariesoACP is an iterative heuristic network optimization algorithm with intelligent learning capabilities, referred to as the Intelligent MNO Algorithm here.…”

Section: B Mobile Network Optimization Algorithmsmentioning

confidence: 99%

Asynchronous distributed parallelization of mobile network optimization algorithms

Megson

Cadenas

2013

Wireless VITAE 2013

View full text Add to dashboard Cite

It has been years since the introduction of the Dynamic Network Optimization (DNO) concept, yet the DNO development is still at its infant stage, largely due to a lack of breakthrough in minimizing the lengthy optimization runtime. Our previous work, a distributed parallel solution, has achieved a significant speed gain. To cater for the increased optimization complexity pressed by the uptake of smartphones and tablets, however, this paper examines the potential areas for further improvement and presents a novel asynchronous distributed parallel design that minimizes the inter-process communications. The new approach is implemented and applied to real-life projects whose results demonstrate an augmented acceleration of 7.5 times on a 16-core distributed system compared to 6.1 of our previous solution. Moreover, there is no degradation in the optimization outcome. This is a solid sprint towards the realization of DNO.

show abstract

On the Probability of Finding Local Minima in Optimization Problems

Cited by 18 publications

References 11 publications

Optimization in “self-modeling” complex adaptive systems

Optimization in “self-modeling” complex adaptive systems

Attraction area of minima in quadratic binary optimization

Asynchronous distributed parallelization of mobile network optimization algorithms

Contact Info

Product

Resources

About