A block coordinate variable metric linesearch based proximal gradient method

Bonettini, Silvia; Prato, Marco; Rebegoldi, Simone

doi:10.1007/s10589-018-0011-5

Cited by 25 publications

(13 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, I k . Nevertheless, the approximation function (20) may be easier to optimize than (21) as the component functions are separable and each component function is a scalar function. This reflects the universal tradeoff between the number of iterations and the complexity per iteration.…”

Section: Algorithmmentioning

confidence: 99%

“…On the convergence speed. The mild assumptions on the approximation functions allow us to design an approximation function that exploits the original problem's structure (such as the partial convexity in (20)- (21)) and this leads to faster convergence. The use of line search also attributes to a faster convergence than decreasing stepsizes used in literature, for example [15,18].…”

Section: Algorithmmentioning

confidence: 99%

“…However, the solution accuracy is specified by an error bound which is difficult to verify in practice. A different approach is adopted in [20,21] where the optimization problem (2) is solved inexactly by running the (proximal) gradient projection algorithm for a finite number of iterations. Nevertheless, its convergence is only established for the specific application in nonnegative matrix factorization in [20] and the use of the (proximal) gradient projection can be restrictive.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization

Yang

Pesavento

Luo

et al. 2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

In this paper, we propose an inexact block coordinate descent algorithm for large-scale nonsmooth nonconvex optimization problems. At each iteration, a particular block variable is selected and updated by solving the original optimization problem with respect to that block variable inexactly. More precisely, a local approximation of the original optimization problem is solved. The proposed algorithm has several attractive features, namely, i) high flexibility, as the approximation function only needs to be strictly convex and it does not have to be a global upper bound of the original function; ii) fast convergence, as the approximation function can be designed to exploit the problem structure at hand and the stepsize is calculated by the line search; iii) low complexity, as the approximation subproblems are much easier to solve and the line search scheme is carried out over a properly constructed differentiable function; iv) guaranteed (subsequence) convergence to a stationary point, even when the objective function does not have a Lipschitz continuous gradient. Interestingly, when the approximation subproblem is solved by a descent algorithm, (subsequence) convergence to a stationary point is still guaranteed even if the approximation subproblem is solved inexactly by terminating the descent algorithm after a finite number of iterations. These features make the proposed algorithm suitable for large-scale problems where the dimension exceeds the memory and/or the processing capability of the existing hardware. These features are also illustrated by several applications in signal processing and machine learning, for instance, network anomaly detection and phase retrieval.

show abstract

Section: Algorithmmentioning

confidence: 99%

Section: Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization

Yang

Pesavento

Luo

et al. 2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…This algorithm uses the update of the gradient direction to replace the calculation of the orthogonal projection, which reduces the computational complexity of the greedy pursuit algorithms. Their successors include the Newton pursuit (NP) [19] algorithm, the conjugate gradient pursuit (CGP) [20] algorithm, the approximate conjugate gradient pursuit (ACGP) [21] algorithm and the variable metric method-based gradient pursuit (VMMGP) [22] algorithm. These methods reduce the computational complexity and storage space of the traditional greedy algorithm in terms of the large-scale recovery problem but the reconstruction performance still requires improvement.…”

Section: Introductionmentioning

confidence: 99%

Stochastic Gradient Matching Pursuit Algorithm Based on Sparse Estimation

Zhao

Liu

2019

Electronics

View full text Add to dashboard Cite

The stochastic gradient matching pursuit algorithm requires the sparsity of the signal as prior information. However, this prior information is unknown in practical applications, which restricts the practical applications of the algorithm to some extent. An improved method was proposed to overcome this problem. First, a pre-evaluation strategy was used to evaluate the sparsity of the signal and the estimated sparsity was used as the initial sparsity. Second, if the number of columns of the candidate atomic matrix was smaller than that of the rows, the least square solution of the signal was calculated, otherwise, the least square solution of the signal was set as zero. Finally, if the current residual was greater than the previous residual, the estimated sparsity was adjusted by the fixed step-size and stage index, otherwise we did not need to adjust the estimated sparsity. The simulation results showed that the proposed method was better than other methods in terms of the aspect of reconstruction percentage in the larger sparsity environment.

show abstract

“…In [1], using the theory about the functions satisfying the Kurdyka Lojasiewicz assumption, a block coordinate approach for minimizing the sum of a differentiable, not necessarily convex, function and non-smooth block separable terms is addressed. This problem arises, for example, in fluorescence microscopy, in emission tomography and in optical astronomy, where we have to perform blind deconvolution of images corrupted by Poisson noise and both the point spread function and the image have to be recovered.…”

mentioning

confidence: 99%

Introduction to the special issue for SIMAI 2016

Ruggiero

Toraldo

2018

Comput Optim Appl

View full text Add to dashboard Cite

In 2016 the biennial congress of the Italian Society of Industrial and Applied Mathematics (SIMAI) was held at Politecnico di Milano (Italy). Promoting collaboration between mathematicians, industrial practitioners and management scientists is among the main goals of SIMAI. In Milan, a number of minisymposia were dedicated to applications and computational aspects of numerical optimization. They were motivated by the historically established interplay between numerical optimization and application problems [3,9], aimed at improving the design, planning and/or performances of processes in industry or management. More recently, optimization has assumed great importance for the extraction of information from very large data sets and, in particular, for signal, image and learning processing in a myriad of application areas. In the optimization problems arising in many real life problems, the design of effective algorithms is a challenging task, either because of the high computational cost of evaluating objectives and constraints, or because of the nonlinearity, multimodality, discontinuity and uncertainty in the data and in the models. Thus the effective solution of practically significant problems is based on a blend of theoretical and computational issues. The contributions from the SIMAI conference included in this special issue are far from giving a complete picture of the mutual impact between numerical optimization and real world problems. Nevertheless, they span a diverse range of topics and applications, and in particular they show that even mathematical issues, that at first glance appear to be very theoretical, can have application-related aspects. This is the case for instance, of the generalization of the Féjer sequences in a real Hilbert space considered in [5]. The main novelty with respect to the standard Valeria Ruggiero and Gerardo Toraldo are the Members of INdAM Research group GNCS, Italy..

show abstract

A block coordinate variable metric linesearch based proximal gradient method

Cited by 25 publications

References 48 publications

Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization

Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization

Stochastic Gradient Matching Pursuit Algorithm Based on Sparse Estimation

Introduction to the special issue for SIMAI 2016

Contact Info

Product

Resources

About