s-Step Enlarged Krylov Subspace Conjugate Gradient Methods

In this paper, a new subspace gradient method is proposed in which the search direction is determined by solving an approximate quadratic model in which a simple symmetric matrix is used to estimate the Hessian matrix in a three-dimensional subspace. The obtained algorithm has the ability to automatically adjust the search direction according to the feedback from experiments. Under some mild assumptions, we use the generalized line search with non-monotonicity to obtain remarkable results, which not only establishes the global convergence of the algorithm for general functions, but also R-linear convergence for uniformly convex functions is further proved. The numerical performance for both the traditional test functions and image restoration problems show that the proposed algorithm is efficient.

show abstract

“…When β k = β DY k , using the same method, we can get (41). Now, if d k is calculated by Equations ( 19) and (21), where…”

Section: Convergence Analysismentioning

confidence: 99%

“…For related research, readers can refer to [17]. More research on the use of subspace technology to construct different methods is still in progress [18][19][20][21][22]. The outline of this article is as follows: in Section 2, we give preliminary information.…”

Section: Introductionmentioning

confidence: 99%

A Dynamically Adjusted Subspace Gradient Method and Its Application in Image Restoration

Huo

Xia

et al. 2021

Symmetry

View full text Add to dashboard Cite

show abstract

“…There have been a number of suggested algorithms for addressing the imbalance in computation and communication within Krylov methods, including communication avoidance [8,19], overlapping communication and computation [10], and delaying communication at the cost of performing more computation [9]. Most recently, there has been work on reducing iterations to convergence via increasing the amount of computation per iteration, and ultimately, the amount of data communicated [14,22,23]. These approaches have been successful…”

Section: Introductionmentioning

confidence: 99%

Performance Analysis and Optimal Node-Aware Communication for Enlarged Conjugate Gradient Methods

Lockhart¹,

Bienz²,

Gropp³

et al. 2022

Preprint

View full text Add to dashboard Cite

Krylov methods are a key way of solving large sparse linear systems of equations, but suffer from poor strong scalabilty on distributed memory machines. This is due to high synchronization costs from large numbers of collective communication calls alongside a low computational workload. Enlarged Krylov methods address this issue by decreasing the total iterations to convergence, an artifact of splitting the initial residual and resulting in operations on block vectors. In this paper, we present a performance study of an Enlarged Krylov Method, Enlarged Conjugate Gradients (ECG), noting the impact of block vectors on parallel performance at scale. Most notably, we observe the increased overhead of point-to-point communication as a result of denser messages in the sparse matrix-block vector multiplication kernel. Additionally, we present models to analyze expected performance of ECG, as well as, motivate design decisions. Most importantly, we introduce a new point-to-point communication approach based on node-aware communication techniques that increases efficiency of the method at scale.

show abstract

Developing variable s-step CGNE and CGNR algorithms for non-symmetric linear systems

Kaveh,

Hajarian,

Chronopoulos

2024

Journal of the Franklin Institute

View full text Add to dashboard Cite

s-Step Enlarged Krylov Subspace Conjugate Gradient Methods

Cited by 4 publications

References 17 publications

A Dynamically Adjusted Subspace Gradient Method and Its Application in Image Restoration

A Dynamically Adjusted Subspace Gradient Method and Its Application in Image Restoration

Performance Analysis and Optimal Node-Aware Communication for Enlarged Conjugate Gradient Methods

Developing variable s-step CGNE and CGNR algorithms for non-symmetric linear systems

Contact Info

Product

Resources

About