Network flows as least squares solvers for linear equations

Abstract

This paper presents a first-order distributed continuous-time algorithm for computing the leastsquares solution to a linear equation over networks. Given the uniqueness of the solution, with nonintegrable and diminishing step size, convergence results are provided for fixed graphs. The exact rate of convergence is also established for various types of step size choices falling into that category. For the case where non-unique solutions exist, convergence to one such solution is proved for constantly connected switching graphs with square integrable step size, and for uniformly jointly connected switching graphs under the boundedness assumption on system states. Validation of the results and illustration of the impact of step size on the convergence speed are made using a few numerical examples. * A preliminary version [11] of this work was presented at the 56th IEEE Conference on Decision and Control, December 12-15, 2017 in Melbourne, Australia.network only has access to one or a few of the individual linear equations making up the full system due to security issues or memory limitation, and is only permitted to interact with a subset of the other agents. A number of contributions have been made to the development of distributed solvable linear equation solvers, where simple first-order distributed algorithms, in continuous-time or discrete-time [1,8,9,12,15,16,23,27,30], manage to deliver satisfactory solutions even for switching network structures.As is known to all, however, another frequent case in practical problems is concerned with non-solvable linear equations, in which we often seek a least-squares solution by minimizing the associated objective function.However, it seems a rather challenging problem in developing distributed least-squares solvers for network linear equations, due to the mismatch between individual linear equations at each node and the network least-squares solution. Despite the difficulties, there exist a few distributed algorithms developed for the least-squares problem using different approaches, such as second-order algorithms [3,10,28,29], state expansion [16] and the high gain consensus gain method [23]. Second-order distributed least-squares solvers [3,10,28,29] generally can produce good convergence performance, however, they rely on restricted network structures and demand higher communication and storage capacities. The state expansion method [16] is based on enlarging the state dimension and then applying the existing methods for linear equations with exact solutions directly, but a negative feature is that the nodes must have access to more knowledge than their own linear equations. It was shown in [23] that first-order algorithms for exact solutions can be adapted to the least-squares case by a high consensus gain, but only in an approximate sense.In this paper, we propose a first-order continuous-time flow for the least-squares problems of network linear equations, in which each agent keeps averaging the state with its neighbors' and at the same time descends along the neg...

Utility of Edge-wise Funnel Coupling for Asymptotically Solving Distributed Consensus Optimization

“…As distributed algorithms have been developed in various fields of study so as to divide a large computational problem into small-scale computations, finding the least-squares solution of a given large linear equation in a distributed manner has been tackled in recent years [22]- [25]. Consider the equation…”

Finite-Time Distributed Linear Equation Solver for Solutions With Minimum $l_1$-Norm

“…A significant amount of effort in the control community has recently been given to distributed algorithms for solving linear equations over multi-agent networks, in which each agent only knows part of the equation and controls a state vector that can be looked at as an estimate of the solution of the overall linear equations [1][2][3][4][5]. Numerous extensions along this direction include achieving solutions with the minimum Euclidean norm [6,7], elimination of the initialization step [8], reduction of state vector dimension by utilizing the sparsity of the linear equation [9] and achieving least square solutions [10][11][12][13][14][15]. All these algorithms yield asymptotic convergence, but require an infinite number of sensing or communication events.…”

Data Rates for Network Linear Equations

“…In the context of parallel computation, computer scientists aimed to develop algorithms that eventually compute part entries of the solutions [10,11,12,13,14]. On the other hand, in view of distributed gradient optimization [15,16,17], distributed algorithms that compute the entire solution vector at each node were also proposed for both discrete-time and continuous-time node dynamics [18,6,19,20,21,22,23,24,25]. In fact, when exact solutions exist for the linear equations, such first-order distributed solvers were generalized versions of the so-called alternation projection algorithms pioneered by von Neumann [26,15,27].…”

“…In fact, when exact solutions exist for the linear equations, such first-order distributed solvers were generalized versions of the so-called alternation projection algorithms pioneered by von Neumann [26,15,27]. When no exact solution exists and one considers least-squares solutions, higher-order algorithms or algorithms using properly selected square-summable diminishing step-sizes are needed [21,22].…”

“…It can be easily concluded from the analysis in [20,22] that the following statements hold for the algorithm (3):…”