Distributed least squares solver for network linear equations

Abstract

In this paper, we study the problem of finding the least square solutions of over-determined linear algebraic equations over networks in a distributed manner. Each node has access to one of the linear equations and holds a dynamic state. We first propose a distributed least square solver over connected undirected interaction graphs and establish a necessary and sufficient on the step-size under which the algorithm exponentially converges to the least square solution. Next, we develop a distributed least square solver over strongly connected directed graphs and show that the proposed algorithm exponentially converges to the least square solution provided the step-size is sufficiently small. Moreover, we develop a finite-time least square solver by equipping the proposed algorithms with a finite-time decentralized computation mechanism. The theoretical findings are validated and illustrated by numerical simulation examples.

FROST—Fast row-stochastic optimization with uncoordinated step-sizes

“…(6a)-(6d) in fact can be derived from an equivalent form of AB after a state transformation on the x k -update; see [42] for details. For applications of the AB algorithm to distributed least squares, see, for instance, [56].…”

Distributed optimization for multi-agent system over unbalanced graphs with linear convergence rate

“…We became aware of a recent work to apply the algorithm to solve linear algebraic function Ax = b in [30] and gave the up bound of the step size under the double stochastic weight matrix. However, as pointed out in the Remark 7 of [30], it is challenge to obtain the up bound of step size in the unbalanced graphs circumstance because of the mix row stochastic and column stochastic matrices. In this paper, for the general strongly convex objective function, the valid step size range is achieved for the unbalanced graphs.…”

Distributed accelerated optimization algorithms: Insights from an ODE

“…Recently, by focusing on the quadratic objective functions, ref. [27] established a necessary and sufficient condition on the step-size for the DIGing algorithm to linearly converge to the exact global optimal solution.…”

“…This motivates our study in this paper. More specifically, in this paper, we first derive a second-order ODE, which is the exact limit of the existing distributed accelerated algorithms, such as the EXTRA [22] and the DIGing algorithm [23][24][25]27]. This can be view as an extension of ref.…”

“…This closely resembles the linear convergence result of the DIGing algorithm established in ref. [27].…”