Asymptotic Properties of $\mathcal{S}$-$\mathcal{AB}$ Method with Diminishing Stepsize

Abstract

The popular AB/push-pull method for distributed optimization problem may unify much of the existing decentralized first-order methods based on gradient tracking technique. More recently, the stochastic gradient variant of AB/Push-Pull method (S-AB) has been proposed, which achieves the linear rate of converging to a neighborhood of the global minimizer when the step-size is constant. This paper is devoted to the asymptotic properties of S-AB with diminishing step-size. Specifically, under the condition that each local objective is smooth and the global objective is strongly-convex, we first present the boundedness of the iterates of S-AB and then show that the iterates converge to the global minimizer with the rate O 1/ √ k . Furthermore, the asymptotic normality of Polyak-Ruppert averaged S-AB is obtained and applications on statistical inference are discussed. Finally, numerical tests are conducted to demonstrate the theoretic results.