MultiStage Approaches for Optimal Offline Checkpointing

Stumm, Philipp; Walther, Andrea

doi:10.1137/080718036

Cited by 37 publications

(32 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the access time to read or write a checkpoint is not negligible in contrast to the assumption frequently made for the development of checkpointing approaches. There are a few contributions that extend the available checkpointing techniques to a hierarchical checkpointing (Aupy, Herrmann, Hovland, & Robert, 2016;Schanen, Marin, Anitescu, & Z, 2016;Stumm & Walther, 2009).…”

Section: Checkpointingmentioning

confidence: 99%

An introduction to algorithmic differentiation

Gebremedhin

Walther

2019

WIREs Data Min & Knowl

Self Cite

View full text Add to dashboard Cite

Algorithmic differentiation (AD), also known as automatic differentiation, is a technology for accurate and efficient evaluation of derivatives of a function given as a computer model. The evaluations of such models are essential building blocks in numerous scientific computing and data analysis applications, including optimization, parameter identification, sensitivity analysis, uncertainty quantification, nonlinear equation solving, and integration of differential equations. We provide an introduction to AD and present its basic ideas and techniques, some of its most important results, the implementation paradigms it relies on, the connection it has to other domains including machine learning and parallel computing, and a few of the major open problems in the area. Topics we discuss include: forward mode and reverse mode of AD, higher‐order derivatives, operator overloading and source transformation, sparsity exploitation, checkpointing, cross‐country mode, and differentiating iterative processes. This article is categorized under: Algorithmic Development > Scalable Statistical Methods Technologies > Data Preprocessing

show abstract

Section: Checkpointingmentioning

confidence: 99%

An introduction to algorithmic differentiation

Gebremedhin

Walther

2019

WIREs Data Min & Knowl

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the sweeps of such F 2 cannot be saved anyway in those checkpointing algorithms (Griewank and Walther 2000, Stumm et al 2009, Wang et al 2009 or else the maximum number of checkpoints could not be reached in this way when the available memory is limited. Under the restriction of a given memory consumption, in another words, no one can make sure which adjoint implementation is of lower computational cost either the maximum number of checkpoints or less in combination of this way.…”

Section: ∇ •mentioning

confidence: 99%

Optimization of the WRFV3.7 adjoint model

Cheng

Liu

Wang

2018

Preprint

View full text Add to dashboard Cite

Abstract. This work focused on a new strategy for productively improving the performance of adjoint models. By using several techniques including the push/pop-free method, careful Input/Output (IO) analysis and the use of the conception of adjoint locality, we reduced the adjoint cost of the Weather Research and Forecasting plus (WRFPLUS) by almost half on different numbers of processors especially with a slight decrease in total memory. Several experiments are conducted using the four-dimensional variational data assimilation (4DVar) method. The results show that the total time cost of running a 4DVar application is decreased by approximately 1/3.

show abstract

“…To test the checkpointing implementation, libadjoint was configured to use the multistage checkpointing algorithm of revolve [49], with 5 checkpoints available in memory and 10 on disk. As described in section 5.1, the use of this checkpointing algorithm is entirely transparent to the DOLFIN user.…”

Section: Examplesmentioning

confidence: 99%

Automated Derivation of the Adjoint of High-Level Transient Finite Element Programs

Farrell¹,

Ham²,

Funke³

et al. 2013

SIAM J. Sci. Comput.

249

233

View full text Add to dashboard Cite

Abstract. In this paper we demonstrate a new technique for deriving discrete adjoint and tangent linear models of finite element models. The technique is significantly more efficient and automatic than standard algorithmic differentiation techniques. The approach relies on a high-level symbolic representation of the forward problem. In contrast to developing a model directly in Fortran or C++, high-level systems allow the developer to express the variational problems to be solved in near-mathematical notation. As such, these systems have a key advantage: since the mathematical structure of the problem is preserved, they are more amenable to automated analysis and manipulation. The framework introduced here is implemented in a freely available software package named dolfin-adjoint, based on the FEniCS Project. Our approach to automated adjoint derivation relies on run-time annotation of the temporal structure of the model, and employs the FEniCS finite element form compiler to automatically generate the low-level code for the derived models. The approach requires only trivial changes to a large class of forward models, including complicated time-dependent nonlinear models. The adjoint model automatically employs optimal checkpointing schemes to mitigate storage requirements for nonlinear models, without any user management or intervention. Furthermore, both the tangent linear and adjoint models naturally work in parallel, without any need to differentiate through calls to MPI or to parse OpenMP directives. The generality, applicability and efficiency of the approach are demonstrated with examples from a wide range of scientific applications. . While deriving the adjoint model associated with a linear stationary forward model is straightforward, the development and implementation of adjoint models for nonlinear or time-dependent forward models is notoriously difficult, for several reasons. First, each nonlinear operator of the forward model must be differentiated, which can be difficult for complex models. Second, the control flow of the adjoint model runs backwards, from the final time to the initial time, and requires access to the solution variables computed during the forward run if the forward problem is nonlinear. Since it is generally impractical for physically relevant simulations to store all variables during the forward run, the adjoint model developer must implement some checkpointing scheme that balances recomputation and storage [17]. The control flow of such a checkpointing scheme must alternate between the solution of forward variables and adjoint variables, and is thus highly

show abstract

MultiStage Approaches for Optimal Offline Checkpointing

Cited by 37 publications

References 12 publications

An introduction to algorithmic differentiation

An introduction to algorithmic differentiation

Optimization of the WRFV3.7 adjoint model

Automated Derivation of the Adjoint of High-Level Transient Finite Element Programs

Contact Info

Product

Resources

About