Avoiding bad steps in Frank-Wolfe variants

Projection free methods on product domains

“…• a local linear convergence rate for any choice of block selection strategy and FW-like direction. This result is obtained under a Kurdyka-Lojasiewicz (KL) property (see, e.g., [3], [6] and [7]) and a tailored angle condition (see, e.g., [34]). Thanks to the way we handle short steps in our framework we are thus able to extend the analysis given for FW variants to the blockcoordinate case and then to close the relevant gap in the theory highlighted in [31].…”

“…Such a flexibility is mainly obtained thanks to the way we perform approximate minimizations in the blocks. At each iteration, after selecting one block at least, we indeed use the Short Step Chain (SSC) procedure described in [34], which skips gradient computations in consecutive short steps until proper conditions are satisfied, to get the approximate minimization done in the selected blocks.…”

“…The first one we consider is a parallel or Jacobi-like strategy (see, e.g., [5]), where the SSC procedure is performed for all blocks. This obviously reduces the computational burden with respect to the use of the SSC in the whole variable space (see, e.g., [34]) and eventually enables to use multicore architectures to perform those tasks in parallel. The second one is the random sampling (see, e.g., [24]), where the SSC procedure is performed at each iteration on a randomly selected subset of blocks.…”

“…Such a greedy rule may make more progress in the objective function, since it uses first order information to choose the right block, but is, in principle, more expensive than the other options we mentioned before (notice that the SSC is performed, at each iteration, for all blocks). Furthermore, we consider the following projection-free strategies: Away-step Frank-Wolfe (AFW), Pairwise Frank-Wolfe (PFW), and Frank-Wolfe method with in face directions (FDFW), see, e.g., [34] and references therein for further details. The AFW and PFW strategies depend on a set of "elementary atoms" A such that C = conv(A).…”

Special issue for SIMAI 2020–2021: large-scale optimization and applications

“…long sequence of very short steps. This problems is addressed in [12], by proposing a theoretically sound algorithmic framework aimed to rule out slow convergence rates due to a large number of bad steps. The approach could have a meaningful impact on the effective use of FW type methods in the general framework of practical applications.…”