Use of Direct Solvers in TFETI Massively Parallel Implementation

Abstract

Abstract. The FETI methods blend iterative and direct solvers. The dual problem is solved iteratively using e.g. CG method; in each iteration, the auxiliary problems related to the application of an unassembled system matrix (subdomain problems' solutions and projector application in dual operator) are solved directly. The paper deals with the comparison of the direct solvers available in PETSc on the Cray XE6 machine HECToR (PETSc, MUMPS, SuperLU) regarding their performance in the two most time consuming actions in TFETI -the pseudoinverse application and the coarse problem solution. For the numerical experiments, our novel TFETI implementation in FLLOP (FETI Light Layer on top of PETSc) library was used.

Parallel strategies for solving the FETI coarse problem in the PERMON toolbox

“…We have suggested and compared several strategies for parallel CP solution [11], [9]. The explicit orthonormalization approach starts to fail when the nullspace is large (thousands).…”

On the scalability of inexact balancing domain decomposition by constraints with overlapped coarse/fine corrections

“…In typical DD implementations, this produces an unacceptable parallel efficiency loss, since all the cores not involved in the coarse solver computation are idling (see Figure 1). One obvious strategy to improve scalability is to reduce the wall-clock time spent at the coarse solver by using, e.g., a MPI-distributed sparse direct solver like MUMPS [9] (see [10] for BDDC and [11] for FETI-DP). However, this approach only mitigates the problem.…”

Simultaneous Input and Output Matrix Partitioning for Outer-Product--Parallel Sparse Matrix-Matrix Multiplication

“…Introduction. Sparse matrix-matrix multiplication (SpGEMM) is a kernel operation in a wide variety of scientific applications such as finite element simulations based on domain decomposition [3,22], molecular dynamics (MD) [15,16,17,25,28,29,32,36], and linear programming (LP) [7,8,26], all of which utilize parallel processing technology to reduce execution times. Among these applications, below we exemplify three methods/codes from which we select realistic SpGEMM instances.…”

“…Sparse matrix-matrix multiplication (SpGEMM) is a kernel operation in a wide variety of scientific applications such as finite element simulations based on domain decomposition [3,22], molecular dynamics (MD) [15,16,17,25,28,29,32,36], and linear programming (LP) [7,8,26], all of which utilize parallel processing technology to reduce execution times. Among these applications, below we exemplify three methods/codes from which we select realistic SpGEMM instances.In finite element application fields, finite element tearing and interconnecting (FETI) [3,22] type domain decomposition methods are used for numerical solution of engineering problems. In this application, the SpGEMM computation GG T is performed, where G = R T B T , R is the block diagonal basis of the stiffness matrix, and B is the signed matrix with entries −1, 0, 1 describing the subdomain interconnectivity.…”

“…In finite element application fields, finite element tearing and interconnecting (FETI) [3,22] type domain decomposition methods are used for numerical solution of engineering problems. In this application, the SpGEMM computation GG T is performed, where G = R T B T , R is the block diagonal basis of the stiffness matrix, and B is the signed matrix with entries −1, 0, 1 describing the subdomain interconnectivity.…”