A Majorize-Minimize Memory Gradient method for complex-valued inverse problems

Florescu, Anisia; Chouzenoux, Émilie; Pesquet, Jean-Christophe; Ciuciu, Philippe; Ciochină, Silviu

doi:10.1016/j.sigpro.2013.09.026

Cited by 35 publications

(33 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…was observed to lead to fast convergence on several examples in the field of signal and image restoration [21,66].…”

Section: Subspace Acceleration Strategymentioning

confidence: 99%

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

Chouzenoux¹,

Pesquet²

2017

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g. a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both non-adaptive and adaptive filter identification problems.

show abstract

“…was observed to lead to fast convergence on several examples in the field of signal and image restoration [21,66].…”

Section: Subspace Acceleration Strategymentioning

confidence: 99%

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

Chouzenoux¹,

Pesquet²

2017

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…0 (under some technical assumptions) [14]. Among the class of possible smoothed`0 functions, the Geman-McClure`2 `0 potential was observed to give good results in a number of applications [17], [14], [18]. It corresponds to the following choice for the function :…”

Section: Introductionmentioning

confidence: 99%

Optimization of a Geman-McClure like criterion for sparse signal deconvolution

Castella

Pesquet

2015

2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)

Self Cite

View full text Add to dashboard Cite

International audienceThis paper deals with the problem of recovering a sparse unknown signal from a set of observations. The latter are obtained by convolution of the original signal and corruption with additive noise. We tackle the problem by minimizing a least-squares fit criterion penalized by a Geman-McClure like potential. The resulting criterion is a rational function, which makes it possible to formulate its minimization as a generalized problem of moments for which a hierarchy of semidefinite programming relaxations can be proposed. These convex relaxations yield a monotone sequence of values which converges to the global optimum. To overcome the computational limitations due to the large number of involved variables, a stochastic block-coordinate descent method is proposed. The algorithm has been implemented and shows promising result

show abstract

“…The subsampling factor R > 1 thus corresponds to an acceleration factor. For a more detailed account on the considered approach, the reader is refered to [99], [100] and the references therein. Reconstruction results are shown in Fig.…”

Section: A Inverse Problemsmentioning

confidence: 99%

Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems

Komodakis

Pesquet

2015

IEEE Signal Process. Mag.

Self Cite

377

283

View full text Add to dashboard Cite

Optimization methods are at the core of many problems in signal/image processing, computer vision, and machine learning. For a long time, it has been recognized that looking at the dual of an optimization problem may drastically simplify its solution. Deriving efficient strategies which jointly brings into play the primal and the dual problems is however a more recent idea which has generated many important new contributions in the last years. These novel developments are grounded on recent advances in convex analysis, discrete optimization, parallel processing, and nonsmooth optimization with emphasis on sparsity issues. In this paper, we aim at presenting the principles of primal-dual approaches, while giving an overview of numerical methods which have been proposed in different contexts. We show the benefits which can be drawn from primal-dual algorithms both for solving large-scale convex optimization problems and discrete ones, and we provide various application examples to illustrate their usefulness. Index TermsConvex optimization, discrete optimization, duality, linear programming, proximal methods, inverse problems, computer vision, machine learning, big data N. Komodakis (corresponding author) and J.-C. Pesquet are with the Optimization [1] is an extremely popular paradigm which constitutes the backbone of many branches of applied mathematics and engineeering, such as signal processing, computer vision, machine learning, inverse problems, and network communications, to mention just a few. The popularity of optimization approaches often stems from the fact that many problems from the above fields are typically characterized by a lack of closed form solutions and by uncertainties. In signal and image processing, for instance, uncertainties can be introduced due to noise, sensor imperfectness, or ambiguities that are often inherent in the visual interpretation. As a result, perfect or exact solutions hardly exist, whereas inexact but optimal (in a statistical or an application-specific sense) solutions and their efficient computation is what one aims at. At the same time, one important characteristic that is nowadays shared by increasingly many optimization problems encountered in the above areas is the fact that these problems are often of very large scale. A good example is the field of computer vision where one often needs to solve low level problems that require associating at least one (and typically more than one) variable to each pixel of an image (or even worse of an image sequence as in the case of video) [2]. This leads to problems that easily can contain millions of variables, which are therefore the norm rather than the exception in this context. Similarly, in fields like machine learning [3], [4], due to the great ease with which data can now be collected and stored, quite often one has to cope with truly massive datasets and to train very large models, which thus naturally lead to optimization problems of very high dimensionality [5]. Of course, a similar situation arises in many other scient...

show abstract

A Majorize-Minimize Memory Gradient method for complex-valued inverse problems

Cited by 35 publications

References 41 publications

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

Optimization of a Geman-McClure like criterion for sparse signal deconvolution

Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems

Contact Info

Product

Resources

About