Active Set Complexity of the Away-Step Frank--Wolfe Algorithm

Abstract

\bfA \bfb \bfs \bft \bfr \bfa \bfc \bft . In this paper, we study active set identification results for the away-step Frank--Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, for nonconvex objectives, a novel O(1/ \surd k) convergence rate result and active set identification for different step sizes (under suitable assumptions on the set of stationary points). By exploiting those results, we also give explicit active set complexity bounds for both strongly convex and nonconvex objectives. While we initially consider the probability simplex as feasible set, in an appendix we show how to adapt some of our results to generic polytopes.\bfK \bfe \bfy \bfw \bfo \bfr \bfd \bfs . surface identification, manifold identification, active set complexity \bfA \bfM \bfS \bfs \bfu \bfb \bfj \bfe \bfc \bft \bfc \bfl \bfa \bfs \bfs \bfi fi\bfc \bfa \bft \bfi \bfo \bfn \bfs . 65K05, 90C06, 90C30 \bfD \bfO \bfI .

An Augmented Lagrangian Method Exploiting an Active-Set Strategy and Second-Order Information

“…It would be of great interest and importance to give a characterization of that neighborhood, in order to bound the maximum number of iterations required by the algorithm to identify the active set. Currently, this is an open problem and we think it may represent a possible line of future research, for example by adapting the complexity results given for ALGENCAN in [7], or extending some results on finite active-set identification given in the literature for specific classes of algorithms [8,10,22].…”

“…We finally terminate the iteration by setting μk+1 as the projection of μ k+1 on a prefixed box, according to (8).…”

Frank–Wolfe and friends: a journey into projection-free first-order optimization methods

“…where we pick the smallest minimizer of the function ϕ for the sake of being welldefined even in rare cases of ties (see, e.g., Bomze et al 2020;Lacoste-Julien and Jaggi 2015). -Armijo line search: the method iteratively shrinks the step size in order to guarantee a sufficient reduction of the objective function.…”

“…It is a classic result that the AFW under some strict complementarity conditions and for strongly convex objectives identifies in finite time the face containing the solution (Guélat and Marcotte 1986). Here we report some explicit bounds for this property proved in Bomze et al (2020). We first assume that C = Δ n−1 , and introduce the multiplier functions…”

“…The above lemma does not require convexity and was applied in Bomze et al (2020) to derive active set identification bounds in several convex and non-convex settings.…”