Adaptive control in roll-forward recovery for extreme scale multigrid

Abstract

With the increasing number of compute components, failures in future exa-scale computer systems are expected to become more frequent. This motivates the study of novel resilience techniques. Here, we extend a recently proposed algorithm-based recovery method for multigrid iterations by introducing an adaptive control. After a fault, the healthy part of the system continues the iterative solution process, while the solution in the faulty domain is re-constructed by an asynchronous on-line recovery. The computations in both the faulty and healthy subdomains must be coordinated in a sensitive way, in particular, both under and over-solving must be avoided. Both of these waste computational resources and will therefore increase the overall time-to-solution. To control the local recovery and guarantee an optimal re-coupling, we introduce a stopping criterion based on a mathematical error estimator. It involves hierarchical weighted sums of residuals within the context of uniformly refined meshes and is well-suited in the context of parallel high-performance computing. The re-coupling process is steered by local contributions of the error estimator. We propose and compare two criteria which differ in their weights. Failure scenarios when solving up to 6.9 · 10 11 unknowns on more than 245 766 parallel processes will be reported on a state-of-the-art peta-scale supercomputer demonstrating the robustness of the method.

Resiliency in numerical algorithm design for extreme scale simulations

“…In combination with the tearing and intersection approach for the recovery, it results in a hybrid approach. In case of a Stokes-type system, yielding after discretization a saddle point problem, the strategy can either be applied on the positive definite Schur complement for the pressure or, as it was done in Huber et al (2019), on the indefinite velocity-pressure system. In that case, an all-at-once multigrid method with an Uzawa-type smoother acting on both solution components turns out to be most efficient (see Drzisga et al, 2018).…”

“…In that case, an all-at-once multigrid method with an Uzawa-type smoother acting on both solution components turns out to be most efficient (see Drzisga et al, 2018). Numerical and algorithmic studies including multiple faults and large-scale problems with more than 5 ⋅ 10 11 degrees of freedom and more than 245,000 cores have been demonstrated (Huber et al, 2016, 2019). The automatic re-coupling strategy is found to be robust with respect to the fault location and size and also handling multiple fault.…”