Quadratic performance of primal-dual methods with application to secondary frequency control of power systems

Simpson-Porco, John W.; Poolla, Bala Kameshwar; Monshizadeh, Nima; Dörfler, Florian

doi:10.1109/cdc.2016.7798532

Cited by 15 publications

(19 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, in many applications including distributed computing over networks [32], [33], coordination in vehicular formations [34], and control of power systems [35], additive white noise is a convenient abstraction for the robustness analysis of distributed control strategies [33] and of first-order optimization algorithms [36], [37]. Motivated by this observation, in this paper we consider the scenario in which a white stochastic noise with zero mean and identity covariance is added to the iterates of standard first-order algorithms: gradient descent, Polyak's heavy-ball method, and Nesterov's accelerated algorithm.…”

Section: Introductionmentioning

confidence: 99%

Variance Amplification of Accelerated First-Order Algorithms for Strongly Convex Quadratic Optimization Problems

Mohammadi

Razaviyayn

Jovanović

2018

2018 IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

We study the robustness of accelerated first-order algorithms to stochastic uncertainties in gradient evaluation.Specifically, for unconstrained, smooth, strongly convex optimization problems, we examine the mean-square error in the optimization variable when the iterates are perturbed by additive white noise. This type of uncertainty may arise in situations where an approximation of the gradient is sought through measurements of a real system or in a distributed computation over network. Even though the underlying dynamics of first-order algorithms for this class of problems are nonlinear, we establish upper bounds on the mean-square deviation from the optimal value that are tight up to constant factors. Our analysis quantifies fundamental trade-offs between noise amplification and convergence rates obtained via any acceleration scheme similar to Nesterov's or heavy-ball methods. To gain additional analytical insight, for strongly convex quadratic problems we explicitly evaluate the steady-state variance of the optimization variable in terms of the eigenvalues of the Hessian of the objective function. We demonstrate that the entire spectrum of the Hessian, rather than just the extreme eigenvalues, influence robustness of noisy algorithms. We specialize this result to the problem of distributed averaging over undirected networks and examine the role of network size and topology on the robustness of noisy accelerated algorithms. the steady-state mean error in the objective value and it was shown that, for the same convergence rate, Nesterov-like method with properly-selected parameters can be more robust than gradient descent. This is not surprising because gradient descent can be viewed as a special case of Nesterov's method with a zero momentum parameter. This observation was used in [41] to design an optimal multi-stage algorithm that does not require information about variance of the noise. In this paper, however, we focus on the variance amplification of the iterates (and not the function value) and discuss the connections between the two robustness measures. We show that any choice of parameters for Nesterov's or heavy-ball methods that yields an accelerated convergence rate increases variance amplification relative to gradient descent. More precisely, for the problem with the condition number κ, an algorithm with accelerated convergence rate of at least 1 − c/ √ κ, where c is a positive constant, increases the variance amplification in the iterates by a factor of √ κ. The robustness problem was also studied in [42] where the authors show similar behavior of Nesterov's method and gradient descent in an asymptotic regime in which the stepsize May 28, 2019 DRAFT 4) We extend our analysis from quadratic objective functions to general strongly convex problems. We borrow an approach based on linear matrix inequalities from control theory to establish upper bounds on the noise amplification of both gradient descent and Nesterov's accelerated algorithm. Furthermore, for any given condition number, we demonstrate th...

show abstract

Section: Introductionmentioning

confidence: 99%

Variance Amplification of Accelerated First-Order Algorithms for Strongly Convex Quadratic Optimization Problems

Mohammadi

Razaviyayn

Jovanović

2018

2018 IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

show abstract

“…Note that the broadcast averaging algorithm does not scale with the system size, further for sufficiently large networks and moderate damping coefficients, the performance improvement in terms of disturbance rejection over the plain vanilla gradient ascent algorithm in [22] is significant.…”

Section: Theorem 31 (Performance Of Broadcast Algorithm)mentioning

confidence: 99%

“…The special case (ii) indicates that the H 2 norm can be suppressed by increasing γ sufficiently. Note in particular that in the high-gain limit (22), the H 2 norm is independent of system size and identical to the broadcast performance (8). This appears to be a fundamental difference between the averaging-based controllers and the open-loop primal-dual controller (12).…”

Section: Theorem 43 (Distributed Averaging Performance)mentioning

confidence: 99%

Quadratic Performance Analysis of Secondary Frequency Controllers

Poolla

Simpson-Porco

Monshizadeh

et al. 2019

2019 IEEE 58th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

This paper investigates the input-output performance of secondary frequency controllers through the controltheoretic notion of H2 norms. We consider a quadratic objective accounting for the cost of reserve procurement and provide exact analytical formulae for the performance of continuoustime aggregated averaging controllers. Then, we contrast it with distributed averaging controllers-seeking optimality conditions such as identical marginal costs-and primal-dual controllers which have gained attention as systematic techniques to design distributed algorithms solving convex optimization problems. Our conclusion is that while the performance of aggregated averaging controllers, such as gather & broadcast, is independent of the system size and driven predominantly by the control gain, the plain vanilla closed-loop primal-dual controllers scale poorly with size and do not offer any improvement over feedforward primal-dual controllers. Finally, distributed averagingbased controllers scale sub-linearly with size and are independent of system size in the high-gain limit. J. W. Simpson-Porco is with the

show abstract

“…We dedicate Section IV to present a relevant power systems example where we present the AGC problem in a PD form then illustrate the error bounds derived in Section III. Note that our results can apply to nonautonomous PD dynamics with general objective functions, while most recent related work deals with autonomous PD dynamics [13] or quadratic objective functions [14]. The ISS analysis in [15], which characterizes error bounds to fixed saddle points, is relevant to the estimates derived in this work.…”

Section: B Contributionsmentioning

confidence: 99%

“…Matrix E describes the network topology, B the rescaled line susceptances, D the rescaled damping ratios, and k the secondary control gains. The PD formż = −∂ ω , ∂ p E , ∂ uagc associated with the Lagrangian function (30) recovers frequency dynamics and the simplified AGC [14], [25]. For the perturbed system, we consider an extension of the model with additional turbine delays modeled asu agc = to theû agc , rather than the AGC effort u agc .…”

Section: Numerical Simulationsmentioning

confidence: 99%

Contraction and Robustness of Continuous Time Primal-Dual Dynamics

Nguyen

Turitsyn

et al. 2018

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

The Primal-Dual (PD) algorithm is widely used in convex optimization to determine saddle points. While the stability of the PD algorithm can be easily guaranteed, strict contraction is nontrivial to establish in most cases. This work focuses on continuous, possibly non-autonomous PD dynamics arising in a network context, in distributed optimization, or in systems with multiple time-scales. We show that the PD algorithm is indeed strictly contracting in specific metrics and analyze its robustness establishing stability and performance guarantees for different approximate PD systems. We derive estimates for the performance of multiple time-scale multi-layer optimization systems, and illustrate our results on a primal-dual representation of the Automatic Generation Control of power systems.

show abstract

Quadratic performance of primal-dual methods with application to secondary frequency control of power systems

Cited by 15 publications

References 23 publications

Variance Amplification of Accelerated First-Order Algorithms for Strongly Convex Quadratic Optimization Problems

Variance Amplification of Accelerated First-Order Algorithms for Strongly Convex Quadratic Optimization Problems

Quadratic Performance Analysis of Secondary Frequency Controllers

Contraction and Robustness of Continuous Time Primal-Dual Dynamics

Contact Info

Product

Resources

About