On solving constrained optimization problems with neural networks: a penalty method approach

Abstract

Deals with the use of neural networks to solve linear and nonlinear programming problems. The dynamics of these networks are analyzed. In particular, the dynamics of the canonical nonlinear programming circuit are analyzed. The circuit is shown to be a gradient system that seeks to minimize an unconstrained energy function that can be viewed as a penalty method approximation of the original problem. Next, the implementations that correspond to the dynamical canonical nonlinear programming circuit are examined. It is shown that the energy function that the system seeks to minimize is different than that of the canonical circuit, due to the saturation limits of op-amps in the circuit. It is also noted that this difference can cause the circuit to converge to a different state than the dynamical canonical circuit. To remedy this problem, a new circuit implementation is proposed.

A cellular neural network and utility-based radio resource scheduler for multimedia CDMA communication systems

“…Also, the output ⃗ ( ) can mostly converge to a reasonable set of stable output of the CNN processor in a percentage of about 99.9%. On the other hand, the convergence speed of CNN processor would be in few miniseconds range [15], [18] since the CNN has an architecture that all neurons are in the same structure, which makes the CNN be suitable for VLSI implementation. Fig.…”

“…An energy function at time which decreases along the trajectories of (17), denoted by ℰ( ), is generally expressed by [13], as (18). At the stable state, outputs of neurons will arrive at an equilibrium with the minimum energy function.…”

“…It determines an optimal normalized radio resource assignment vector for connections in multimedia CDMA cellular systems by minimizing the system cost function, which is in terms of the overall system utility function under system constraints of maximum transmission power, minimum spreading factor, and remaining queue length. The architecture of the CNN processor is constructed by a Lyapunov method [18], [19], via mapping the system cost function to a proper energy function. It is designed in a two-layered conguration, which consists of a decision layer and an output layer, to reduce the number of inter-connections in the CNN.…”

“…According to the trajectory of the energy function, we can construct the architecture of the desired CNN processor to t the dened scheduling problem. Notice that it can be proven that the designed CNN processor can be with good stability and convergence [23], based on the principle of Lyapunov and the stability theorem of CNN [18], [19]. , ( ) consists of recurrent inputs, external inputs, and a bias current.…”

A Feedback Neural Network for Solving Nonlinear Programming Problems with Hybrid Constraints

“…. Neural network model for solving (9) was developed in [15]. Its dynamical equation is described by It has two layers and because of an additional nonlinear term, it is more complex in structure than the proposed neural network model (10).…”