Optimal Gradient Tracking for Decentralized Optimization

Song, Zhuoqing; Shi, Lei; Pu, Shi; Yan, Ming

doi:10.48550/arxiv.2110.05282

Cited by 1 publication

(1 citation statement)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The condition number of P K (W) equals O(1) due to the specific structure of the polynomial (see [57]). Moreover, loopless Chebyshev acceleration proposed in [59] is achieved without explicit multi-step consensus.…”

Section: Multi-step Consensusmentioning

confidence: 99%

Penalty-Based Method for Decentralized Optimization over Time-Varying Graphs

Rogozin

Gasnikov

2020

Optimization and Applications

View full text Add to dashboard Cite

We consider a distributed stochastic optimization problem that is solved by a decentralized network of agents with only local communication between neighboring agents. The goal of the whole system is to minimize a global objective function given as a sum of local objectives held by each agent. Each local objective is defined as an expectation of a convex smooth random function and the agent is allowed to sample stochastic gradients for this function. For this setting we propose the first accelerated (in the sense of Nesterov's acceleration) method that simultaneously attains optimal up to a logarithmic factor communication and oracle complexity bounds for smooth strongly convex distributed stochastic optimization. We also consider the case when the communication graph is allowed to vary with time and obtain complexity bounds for our algorithm, which are the first upper complexity bounds for this setting in the literature.

show abstract

Section: Multi-step Consensusmentioning

confidence: 99%