Gradient Methods with Adaptive Step-Sizes

Zhou, Bin; Gao, Liquan; Dai, Yu‐Hong

doi:10.1007/s10589-006-6446-0

Cited by 184 publications

(176 citation statements)

References 22 publications

Supporting

Mentioning

166

Contrasting

Order By: Relevance

“…In order to prove (19)- (20), let us reason by induction on j. Using the triangle inequality, (14) with k = 0, the monotonicity of {Ψ(x (k) )} k∈N and (13) we have…”

Section: Convergence Resultsmentioning

confidence: 99%

“…Finally, direct use of (16) shows that (20) holds with j = 1. By induction, suppose that (19)- (20) hold for some j ≥ 1.…”

Section: Convergence Resultsmentioning

confidence: 99%

“…The SGP method is a variable metric forward-backward algorithm [11,12] which has been exploited in the last years for the solution of different real-world inverse problems [5,6,7,16,17,18]. The main difference between SGP and the standard forward-backward schemes is the presence of two independent parameters α k and λ k with a complete different role: while the last one is automatically computed with the Armijo condition (2) to guarantee the sufficient decrease of the objective function, the first one can be chosen to improve the actual convergence rate of the method, exploiting thirty years of literature in numerical optimization [3,13,19,20]. We also remark that, unlike the schemes presented e.g.…”

Section: The Scaled Gradient Projection Methodsmentioning

confidence: 99%

See 2 more Smart Citations

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Prato

Bonettini

Loris

et al. 2016

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Abstract. The scaled gradient projection (SGP) method is a first-order optimization method applicable to the constrained minimization of smooth functions and exploiting a scaling matrix multiplying the gradient and a variable steplength parameter to improve the convergence of the scheme. For a general nonconvex function, the limit points of the sequence generated by SGP have been proved to be stationary, while in the convex case and with some restrictions on the choice of the scaling matrix the sequence itself converges to a constrained minimum point. In this paper we extend these convergence results by showing that the SGP sequence converges to a limit point provided that the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain and its gradient is Lipschitz continuous.

show abstract

“…In order to prove (19)- (20), let us reason by induction on j. Using the triangle inequality, (14) with k = 0, the monotonicity of {Ψ(x (k) )} k∈N and (13) we have…”

Section: Convergence Resultsmentioning

confidence: 99%

“…Finally, direct use of (16) shows that (20) holds with j = 1. By induction, suppose that (19)- (20) hold for some j ≥ 1.…”

Section: Convergence Resultsmentioning

confidence: 99%

Section: The Scaled Gradient Projection Methodsmentioning

confidence: 99%

See 1 more Smart Citation

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Prato

Bonettini

Loris

et al. 2016

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…The step size of GP is analytically calculated and adaptively changes during the optimization. 30 The high computational efficiency of this GP-BB method has been demonstrated in Refs. 25 and 30 by numerical experiments.…”

Section: Iia Formulation Of Abocs Frameworkmentioning

confidence: 90%

Accelerated barrier optimization compressed sensing (ABOCS) reconstruction for cone-beam CT: Phantom studies

2012

View full text Add to dashboard Cite

Purpose: Recent advances in compressed sensing (CS) enable accurate CT image reconstruction from highly undersampled and noisy projection measurements, due to the sparsifiable feature of most CT images using total variation (TV). These novel reconstruction methods have demonstrated advantages in clinical applications where radiation dose reduction is critical, such as onboard cone-beam CT (CBCT) imaging in radiation therapy. The image reconstruction using CS is formulated as either a constrained problem to minimize the TV objective within a small and fixed data fidelity error, or an unconstrained problem to minimize the data fidelity error with TV regularization. However, the conventional solutions to the above two formulations are either computationally inefficient or involved with inconsistent regularization parameter tuning, which significantly limit the clinical use of CS-based iterative reconstruction. In this paper, we propose an optimization algorithm for CS reconstruction which overcomes the above two drawbacks. Methods:The data fidelity tolerance of CS reconstruction can be well estimated based on the measured data, as most of the projection errors are from Poisson noise after effective data correction for scatter and beam-hardening effects. We therefore adopt the TV optimization framework with a data fidelity constraint. To accelerate the convergence, we first convert such a constrained optimization using a logarithmic barrier method into a form similar to that of the conventional TV regularization based reconstruction but with an automatically adjusted penalty weight. The problem is then solved efficiently by gradient projection with an adaptive Barzilai-Borwein step-size selection scheme. The proposed algorithm is referred to as accelerated barrier optimization for CS (ABOCS), and evaluated using both digital and physical phantom studies. Results: ABOCS directly estimates the data fidelity tolerance from the raw projection data. Therefore, as demonstrated in both digital Shepp-Logan and physical head phantom studies, consistent reconstruction performances are achieved using the same algorithm parameters on scans with different noise levels and/or on different objects. On the contrary, the penalty weight in a TV regularization based method needs to be fine-tuned in a large range (up to seven times) to maintain the reconstructed image quality. The improvement of ABOCS on computational efficiency is demonstrated in the comparisons with adaptive-steepest-descent-projection-onto-convex-sets (ASD-POCS), an existing CS reconstruction algorithm also using constrained optimization. ASD-POCS alternatively minimizes the TV objective using adaptive steepest descent (ASD) and the data fidelity error using projection onto convex sets (POCS). For similar image qualities of the Shepp-Logan phantom, ABOCS requires less computation time than ASD-POCS in MATLAB by more than one order of magnitude. Conclusions: We propose ABOCS for CBCT reconstruction. As compared to other published CSbased algorithms, our method has at...

show abstract

“…The BB algorithm has been much studied because of its remarkable improvement over the SD and OM algorithms ( [25,4,17] and references therein) and proofs of its convergence can be found in [20]. However, a complete explanation of why this simple modification of the SD algorithm improves its performance considerably has not yet been found, although it has been suggested that the improvement is connected to its nonmonotonic convergence, as well as to the fact that it does not produce iterates that get trapped in a low dimensional subspace [4].…”

Section: Preliminariesmentioning

confidence: 99%

Cooperative concurrent asynchronous computation of the solution of symmetric linear systems

2016

View full text Add to dashboard Cite

This paper extends the idea of Brezinski's hybrid acceleration procedure, for the solution of a system of linear equations with a symmetric coefficient matrix of dimension n, to a new context called cooperative computation, involving m agents (m n), each one concurrently computing the solution of the whole system, using an iterative method. Cooperation occurs between the agents through the communication, periodic or probabilistic, of the estimate of each agent to one randomly chosen agent, thus characterizing the computation as concurrent and asynchronous. Every time one agent receives solution estimates from the others, it carries out a least squares computation, involving a small linear system of dimension m, in order to replace its current solution estimate by an affine combination of the received estimates, and the algorithm continues until a stopping criterion is met. In addition, an autocooperative algorithm, in which estimates are updated using affine combinations of current and past estimates, is also proposed. The proposed algorithms are shown to be efficient for certain matrices, specifically in relation to the popular Barzilai-Borwein algorithm, through numerical experiments.

show abstract

Gradient Methods with Adaptive Step-Sizes

Cited by 184 publications

References 22 publications

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Accelerated barrier optimization compressed sensing (ABOCS) reconstruction for cone-beam CT: Phantom studies

Cooperative concurrent asynchronous computation of the solution of symmetric linear systems

Contact Info

Product

Resources

About