Distributed Zero-Order Algorithms for Nonconvex Multi-Agent optimization

Abstract

Distributed multi-agent optimization is the core of many applications in distributed learning, control, estimation, etc. Most existing algorithms assume knowledge of first-order information of the objective and have been analyzed for convex problems. However, there are situations where the objective is nonconvex, and one can only evaluate the function values at finitely many points. In this paper we consider derivative-free distributed algorithms for nonconvex multi-agent optimization, based on recent progress in zero-order optimization. We develop two algorithms for different settings, provide detailed analysis of their convergence behavior, and compare them with existing centralized zero-order algorithms and gradient-based distribution algorithms. Recently there has been increasing interest in zero-order optimization, where one does not have access to the gradient of the objective. Such situations can occur, for example, when only black-box procedures are available for computing the values of the functional characteristics of the problem, or when resource limitations restrict the use of fast or automatic differentiation techniques. Many existing works on zero-order optimization are based on constructing gradient estimators using finitely many function evaluations. [26] proposed and analyzed a single-point gradient estimator, and [27] further studied the convergence rate of single-point zero-order algorithms for highly smooth objectives.[28] proposed two-point gradient estimators and showed that the convergence of the resulting algorithms are comparable with their first-order counterparts.[29] studied two-point gradient estimators in stochastic nonconvex zero-order optimization.[30] and [31] showed that for stochastic zero-order convex optimization with two-point gradient estimators, the optimal rate O( d/N ) is achievable where N denotes the number of function value queries. [32] proposed and analyzed a zero-order stochastic Frank-Wolfe algorithm.Some recent works have also started to combine zero-order and distributed methods.[33] proposed a distributed zero-order algorithm for stochastic nonconvex problems based on the method of multipliers.[34] proposed a zero-order ADMM algorithm for distributed online convex optimization. [35] proposed a distributed zero-order algorithm over random networks and established its convergence for strongly convex objectives.[36] considered distributed zero-order methods for constrained convex optimization. On the other hand, there are still many questions remain to be studied in distributed zero-order optimization, e.g., how zero-order and distributed methods affect the performance of each other and whether their fundamental structural properties could be kept by tuning the way of their combination. This paper aims at providing messages along this line: We propose and analyze two zero-order distributed algorithms for deterministic nonconvex optimization, and compare their convergence rates with their distributed first-order and centralized zero-order counterparts. The first alg...

Distributed Online Convex Optimization With Time-Varying Coupled Inequality Constraints

“…Gradient-free optimization methods have a long history [19] and have an evident advantage since computing a function value is much simpler than computing its gradient. Gradient-free optimization methods have gained renewed interests in recent years, e.g., [20]- [23]. Essentially, bandit online convex optimization is a gradientfree method to solve convex optimization problems.…”

Communication Compression for Distributed Nonconvex Optimization

“…On the other hand, noting that it has been shown in [9], [39], [40] that global optima of nonconvex optimization can be linearly found if the global cost function satisfies the Polyak-Łojasiewicz (P-Ł) condition, another core theoretical question arises.…”

FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing

“…Yet, we must point out that there exist various different alternatives for distributed optimization, such as gossip algorithms [94], as well as some new alternatives including for instance distributed second-order Newton methods [95], distributed zeroorder optimization methods [96] and parallel coordinate descent [97], etc. Depending on the task specification, these alternatives might be more effective than ADMM.…”