On the linear convergence rates of exchange and continuous methods for total variation minimization

Abstract

We analyze an exchange algorithm for the numerical solution total-variation regularized inverse problems over the space M(Ω) of Radon measures on a subset Ω of R d. Our main result states that under some regularity conditions, the method eventually converges linearly. Additionally, we prove that continuously optimizing the amplitudes of positions of the target measure will succeed at a linear rate with a good initialization. Finally, we propose to combine the two approaches into an alternating method and discuss the comparative advantages of this approach.

Linear convergence of accelerated conditional gradient algorithms in spaces of measures

“…After this manuscript was finalized we were made aware of [28], where the authors prove linear convergence of a similar accelerated conditional gradient method for the particular case of H = C = R and G( u M ) = β u M . We note that [28] and the present manuscript were derived independently of each other and differ in certain important aspects. In particular, in contrast to our work, the authors require A k ⊂ A k+1 , i.e.…”

Sparse optimization on measures with over-parameterized gradient descent

“…When Θ is continuous, the one dimensional case can sometimes be dealt with specific algorithms [13,15]. In higher dimensions, the classical algorithms are conditional gradient algorithms (also known as Franck-Wolfe) [11,25,8], moment methods [24,14,27] and adaptive sampling/exchange algorithms [30,29]. Often, these algorithms are complemented with non-convex updates on the particle positions, which considerably improves their behavior.…”

“…Given an initial condition that is close to the optimum and with the same structure (i.e. without over-parameterization), the local convergence for non-convex gradient descent is studied in [56,29]. Wasserstein gradient flows for optimization.…”

“…The second property is satisfied when the kernel associated to the feature function∇φ is positive definite. The last two assumptions unfortunately depend on an a priori unknown object J ν , but are often required to perform analysis of Problem (1) [29,28]. Yet, in some cases, they can be guaranteed to hold, see e.g.…”

When does OMP achieve exact recovery with continuous dictionaries?

“…The conditional gradient method (CGM) has also proven to be applicable to address the BLasso problem [29] and further enhanced with nonconvex local optimization extra steps [33][34][35]. Interestingly, the CGM has been shown to be equivalent to the so-called exchange method in [36,37]. More recently, gradient-flow methods on spaces of measures have also been investigated to address the BLasso problem [38,39].…”