Tensor methods for strongly convex strongly concave saddle point problems and strongly monotone variational inequalities

Abstract

In this paper we propose two p-th order tensor methods for µstrongly-convex-strongly-concave saddle point problems. The first method is based on the assumption of L p -smoothness of the gradient of the objective and it achieves a convergence rate of O((L p R p /µ) 2 p+1 log(µR 2 /ε)), where R is an estimate of the initial distance to the solution. Under additional assumptions of L 1 -, L 2 and L p -smoothness of the gradient of the objective we connect the first method with a locally superlinear converging algorithm and develop a new method with the complexity of O((L p R p /µ)Since we treat saddle-point problems as a particular case of variational inequalities, we also propose two methods for strongly monotone variational inequalities with the same complexity as the described above.

Generalized Optimistic Methods for Convex-Concave Saddle Point Problems

“…Comparison with [OKDG20]. We close the discussion on our iteration complexity by mentioning that similar complexity bounds to (65) and (66) were also established in [OKDG20] but via a very different approach.…”

“…Comparison with [OKDG20]. We close the discussion on our iteration complexity by mentioning that similar complexity bounds to (65) and (66) were also established in [OKDG20] but via a very different approach. Specifically, the authors of that paper proposed to apply a restarting technique on the extragradient-type method in [BL20].…”

“…derivative. Later, the authors in [OKDG20] built upon this work and proposed a restarted highorder mirror-prox method in the strongly-convex-strongly-concave setting, achieving a complexity bound of O(κ 2 p+1 p log(ǫ −1 )). Under the additional assumption that the gradient of f is Lipschitz, they further combined the restarted scheme with the cubic regularized Newton method in [HZZ20], leading to an overall complexity bound of O(κ 2 p+1 p log(κ 1 κ 2 ) + log log(ǫ −1 )).…”