A Majorize–Minimize Strategy for Subspace Optimization Applied to Image Restoration

Abstract

This paper proposes accelerated subspace optimization methods in the context of image restoration. Subspace optimization methods belong to the class of iterative descent algorithms for unconstrained optimization. At each iteration of such methods, a stepsize vector allowing the best combination of several search directions is computed through a multidimensional search. It is usually obtained by an inner iterative second-order method ruled by a stopping criterion that guarantees the convergence of the outer algorithm. As an alternative, we propose an original multidimensional search strategy based on the majorize-minimize principle. It leads to a closed-form stepsize formula that ensures the convergence of the subspace algorithm whatever the number of inner iterations. The practical efficiency of the proposed scheme is illustrated in the context of edge-preserving image restoration.

Efficient Variational Bayesian Approximation Method Based on Subspace Optimization

“…where x k and x k+1 respectively stands for the estimates at kth and (k+1)th iterations, An overview of existing subspace optimization methods [17] shows that D k usually includes a descent direction (e.g. gradient, Newton, truncated Newton direction) and a short history of previous directions.…”

“…Chouzenoux et al [17] have addressed a discussion about the dimension of the subspace through simulation results on several image restoration problems. It is shown that in a Hilbert space, for a super memory gradient subspace (9), taking I = 2 i.e.…”

“…For the aim of developing more efficient methods, we transpose here in the same context the subspace optimization method which has been shown to outperform standard optimization methods, such as gradient or conjugate gradient methods, in terms of rate of convergence in finite dimensional Hilbert spaces [17].…”

“…Moreover, the descent direction can be freely chosen in a subspace of dimension greater than one. This flexibility allows subspace optimization methods to be generally more efficient than conjugate gradient methods [17]. Based on the subspace optimization methods, an improved variational Bayesian optimization method is proposed in this paper.…”

An Iterative Linear Expansion of Thresholds for $\ell_{1}$-Based Image Restoration

“…This strategy can be generalized with the sequential subspace optimization (SESOP) [32], [33] for further accelerations, e.g. PCD-SESOP [11], [33] and PCD-SESOP-MM [34]. Variable splitting technique [35], [36] (also known as separable surrogate functionals (SSF) [33]) provides yet another powerful tool to minimize functions that consist of summation of two terms which are of different nature, e.g.…”