On global and local convergence of half-quadratic algorithms

Abstract

This paper provides original results on the global and local convergence properties of half-quadratic (HQ) algorithms resulting from the Geman and Yang (GY) and Geman and Reynolds (GR) primal-dual constructions. First, we show that the convergence domain of the GY algorithm can be extended with the benefit of an improved convergence rate. Second, we provide a precise comparison of the convergence rates for both algorithms. This analysis shows that the GR form does not benefit from a better convergence rate in general. Moreover, the GY iterates often take advantage of a low cost implementation. In this case, the GY form is usually faster than the GR form from the CPU time viewpoint.

The Equivalence of Half-Quadratic Minimization and the Gradient Linearization Iteration

“…[20,13,21,14,22,23]. Connections of half-quadratic method with other well-known methods have also been explored, most notably, with generalized Weiszfeld's method [25], with statistical EM algorithms [26], with Lagrangian unconstrained optimization in recursive robust fitting [27], with quasi-Newton minimization [15,28] and as a residual steepest descent method [28].…”

“…The convergence of the resulting iterative scheme (16)- (17) was considered under different assumptions on ϕ in [13,21,23] while its speed was analyzed in [19,28].…”

Half Quadratic Analysis for Mean Shift: with Extension to A Sequential Data Mode-Seeking Method

“…Until now, the convergence rate of mean shift has been scarcely addressed. In [3], it is established that the quotientconvergence order of the Gaussian kernel mean shift is generally linear 1 . On the other hand, the convergence rate of HQ is deeply studied in literature [1,14], rather relies on root-convergence 2 factors.…”

“…In [3], it is established that the quotientconvergence order of the Gaussian kernel mean shift is generally linear 1 . On the other hand, the convergence rate of HQ is deeply studied in literature [1,14], rather relies on root-convergence 2 factors. Since our work bridges the gap between mean shift and HQ optimization, the discussion on convergence rate for mean shift is facilitated.…”

“…The motivation for the theoretical part is to justify the current understanding of convex profile based mean shift from the half-quadratic (HQ) optimization viewpoint. HQ is a standard optimization technique in convex analysis [17,1]. By introducing dual variables, non-quadratic convex objective functions can be maximized in a quadratic-like way inside the HQ framework.…”