Asynchronous Distributed Optimization with Heterogeneous Regularizations and Normalizations

Hochhaus, Stefan; Hale, Matthew

doi:10.1109/cdc.2018.8619015

Cited by 5 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A distributed optimization algorithm for convex optimization with local inequality constraints has been studied in [78]. An asynchronous distributed algorithm with heterogeneous regularizations and normalizations is proposed in [79]. A specialized version of the distributed subgradient algorithm for convex feasibility problems, which allows for an infinite number of constraint sets, is presented in [80].…”

Section: Discussion and Referencesmentioning

confidence: 99%

Distributed Optimization for Smart Cyber-Physical Networks

Notarstefano¹,

Notarnicola²,

Camisa³

2019

View full text Add to dashboard Cite

The presence of embedded electronics and communication capabilities as well as sensing and control in smart devices has given rise to the novel concept of cyber-physical networks, in which agents aim at cooperatively solving complex tasks by local computation and communication. Numerous estimation, learning, decision and control tasks in smart networks involve the solution of large-scale, structured optimization problems in which network agents have only a partial knowledge of the whole problem. Distributed optimization aims at designing local computation and communication rules for the network processors allowing them to cooperatively solve the global optimization problem without relying on any central unit. The purpose of this survey is to provide an introduction to distributed optimization methodologies. Principal approaches, namely (primal) consensus-based, duality-based and constraint exchange methods, are formalized. An analysis of the basic schemes is supplied, and state-of-the-art extensions are reviewed.

show abstract

Section: Discussion and Referencesmentioning

confidence: 99%

Distributed Optimization for Smart Cyber-Physical Networks

Notarstefano¹,

Notarnicola²,

Camisa³

2019

View full text Add to dashboard Cite

show abstract

“…Block-based methods have previously been shown to tolerate arbitrarily long delays in both communications and computations in some unconstrained problems [4], [19], [32], eliminating the need to enforce and verify delay boundedness assumptions. For constrained problems of a general form, block-based methods have been paired with primal-dual algorithms with centralized dual updates [15], [17] and/or synchronous primal-updates [21].…”

Section: Introductionmentioning

confidence: 99%

Towards Totally Asynchronous Primal-Dual Convex Optimization in Blocks

Hendrickson

Hale

2020

2020 59th IEEE Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variables, and communications of dual variables. We construct a family of explicit counterexamples to show the need to eliminate asynchronous communication of dual variables, though the other forms of asynchrony are permitted, all without requiring bounds on delays. A first-order primal-dual update law is developed and shown to be robust to asynchrony. We then derive convergence rates to a Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. These convergence rates include an "asynchrony penalty" that we quantify and present ways to mitigate.Numerical results illustrate these developments.

show abstract

Totally Asynchronous Distributed Quadratic Programming with Independent Stepsizes and Regularizations

Ubl

Hale

2019

2019 IEEE 58th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

Quadratic programs arise in robotics, communications, smart grids, and many other applications. As these problems grow in size, finding solutions becomes much more computationally demanding, and new algorithms are needed to efficiently solve them. Targeting large-scale problems, we develop a multi-agent quadratic programming framework in which each agent updates only a small number of the total decision variables in a problem. Agents communicate their updated values to each other, though we do not impose any restrictions on the timing with which they do so, nor on the delays in these transmissions. Furthermore, we allow weak parametric coupling among agents, in the sense that they are free to independently choose their stepsizes, subject to mild restrictions. We show that these stepsize restrictions depend upon a problem's condition number. We further provide the means for agents to independently regularize the problem they solve, thereby improving condition numbers and, as we will show, convergence properties, while preserving agents' independence in selecting parameters. Simulation results are provided to demonstrate the success of this framework on a practical quadratic program. I. INTRODUCTIONConvex optimization problems arise in a diverse array of engineering applications, including signal processing [1], robotics [2], [3], communications [4], machine learning [5], and many others [6]. In all of these areas, problems can become very large as the number of network members (robots, processors, etc.) becomes large. Accordingly, there has arisen interest in solving large-scale optimization problems. A common feature of large-scale solvers is that they are parallelized or distributed among a collection of agents in some way. As the number of agents grows, it can be difficult or impossible to ensure synchrony among distributed computations and communications, and there has therefore arisen interest in distributed asynchronous optimization algorithms.One line of research considers asynchronous optimization algorithms in which agents' communication topologies vary in time. A representative sample of this work includes [7]- [12], and these algorithms all rely on an underlying averaging-based update law, i.e., different agents update the same decision variables and then repeatedly average their iterates to mitigate disagreements that stem from asynchrony. These approaches (and others in the literature) require some form of graph connectivity over intervals of a finite length. In this paper, we are interested in cases in which delay bounds are outside agents' control, e.g., due to environmental hazards and adversarial jamming

show abstract

Asynchronous Distributed Optimization with Heterogeneous Regularizations and Normalizations

Cited by 5 publications

References 21 publications

Distributed Optimization for Smart Cyber-Physical Networks

Distributed Optimization for Smart Cyber-Physical Networks

Towards Totally Asynchronous Primal-Dual Convex Optimization in Blocks

Totally Asynchronous Distributed Quadratic Programming with Independent Stepsizes and Regularizations

Contact Info

Product

Resources

About