Combining Checkpointing and Data Compression to Accelerate Adjoint-Based Optimization Problems

Kukreja, Navjot; Hückelheim, Jan; Louboutin, Mathias; Hovland, Paul; Gorman, Gerard J.

doi:10.1007/978-3-030-29400-7_7

Cited by 6 publications

(17 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we wish to combine these approaches into one three-way trade-off. In previous work [14], we have shown that it is possible to accelerate adjointbased computations, including FWI, by using lossy compression on the checkpoints. For a given checkpoint absolute error tolerance (atol ), compression may or may not accelerate the computation.…”

Section: Contributionsmentioning

confidence: 99%

Lossy Checkpoint Compression in Full Waveform Inversion

Kukreja¹,

Hückelheim²,

Louboutin³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. This paper proposes a new method that combines check- pointing methods with error-controlled lossy compression for large-scale high-performance Full-Waveform Inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can signif- icantly reduce data movement, allowing a reduction in run time as well as peak memory. In the Exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory foot- print during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution.

show abstract

Section: Contributionsmentioning

confidence: 99%

Lossy Checkpoint Compression in Full Waveform Inversion

Kukreja¹,

Hückelheim²,

Louboutin³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In References 16 and 17, check‐pointing was combined with compression techniques for Full‐Waveform Inversion, an adjoint‐based optimization problem commonly used in geophysical exploration. A key difference to these two papers is that in References 16 and 17 the ZFP algorithm was used to compress data computed on structured computational grids.…”

Section: Introductionmentioning

confidence: 99%

Reducing memory requirements of unsteady adjoint by synergistically using check‐pointing and compression

Margetis

Papoutsis‐Kiachagias

Giannakoglou

2022

Numerical Methods in Fluids

View full text Add to dashboard Cite

The unsteady adjoint method used in gradient‐based optimization in 2D and, particularly, 3D industrial problems modeled by unsteady PDEs may have significant storage requirements and/or computational cost. The reason for this is that the backward in time integration of the adjoint equations requires the previously computed instantaneous flow fields to be available at each time‐step. This article proposes remedies to this problem, by extending/upgrading relevant techniques proposed by the group of authors as well as other researchers. Their applicability is wide, even if these remedies are herein demonstrated in shape optimization problems in unsteady fluid mechanics. Check‐pointing is in widespread use as it reduces the memory footprint and CPU cost of the optimization with a controllable computational overhead. Alternatively, flow field time‐series can be stored in a lossless or lossly compressed form. The novelty of this article is the development of a Compressed Coarse‐grained Check‐Pointing strategy for second‐order accurate schemes in time, by optimally combining check‐pointing and lossy compression. The latter includes (a) the incremental Proper Generalized Decomposition (iPGD) algorithm and (b) a hybridization of the iPGD with the ZFP and Zlib algorithms. This is implemented within OpenFOAM, which is used to solve the flow and adjoint equations and conduct the optimization, and assessed in 2D/3D aerodynamic shape optimization problems on unstructured grids. Effectiveness in data reduction, computational cost, and reconstruction accuracy are compared, vis‐à‐vis also to the “standard” binomial check‐pointing technique after adjusting it to second‐order accurate schemes in time.

show abstract

“…The adjoint-state method requires solving the acoustic wave equation forward in time and then storing this solution in memory at all time steps until the adjoint wave equation is solved because the two wavefields must be accessed simultaneously for computing the gradient. Prior studies in the field of geophysics have explored ways to reduce FWI memory consumption through different forms of checkpointing 9,10 and lossy compression 9,11 . In checkpointing approaches, the forward solution of the wave equation is stored only at specific time steps, while the remaining ones are discarded and later recomputed during the adjoint calculation 9 .…”

Section: Introductionmentioning

confidence: 99%

“…Prior studies in the field of geophysics have explored ways to reduce FWI memory consumption through different forms of checkpointing 9,10 and lossy compression 9,11 . In checkpointing approaches, the forward solution of the wave equation is stored only at specific time steps, while the remaining ones are discarded and later recomputed during the adjoint calculation 9 . This, however, can result in significant computational overhead.…”

Section: Introductionmentioning

confidence: 99%

“…This, however, can result in significant computational overhead. For this reason, checkpointing is sometimes combined with other techniques, such as lossy compression, to reduce the size of the stored states and, therefore, the amount of states to be recomputed 9 . Previous studies have also used lossy compression on its own to reduce the size of the stored wavefield, thus avoiding the large computational overhead that characterises checkpointing methods 11 .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computationally efficient full-waveform inversion of the brain using frequency-adaptive grids and lossy compression

Letizia¹,

Cueto²

2021

Preprint

View full text Add to dashboard Cite

A tomographic technique called full-waveform inversion has recently shown promise as a fast, affordable, and safe modality to image the brain using ultrasound. However, its high computational cost and memory footprint currently limit its clinical applicability. Here, we address these challenges through a frequency-adaptive discretisation of the imaging domain and lossy compression techniques. Because full-waveform inversion relies on the adjoint-state method, every iteration involves solving the wave equation over a discretised spatiotemporal grid and storing the numerical solution to calculate gradient updates. The computational cost depends on the grid size, which is controlled by the maximum frequency being modelled. Since the propagated frequency typically varies during the reconstruction, we reduce reconstruction time and memory use by allowing the grid size to change throughout the inversion. Moreover, we combine this approach with multiple lossy compression techniques that exploit the sparsity of the wavefield to further reduce its memory footprint. We explore applying these techniques in the spatial, wavelet, and wave atom domains. Numerical experiments using a human-head model show that our methods lead to a 30% reduction in reconstruction time and up to three orders of magnitude less memory, while negligibly affecting the accuracy of the reconstructions.

show abstract

Combining Checkpointing and Data Compression to Accelerate Adjoint-Based Optimization Problems

Cited by 6 publications

References 16 publications

Lossy Checkpoint Compression in Full Waveform Inversion

Lossy Checkpoint Compression in Full Waveform Inversion

Reducing memory requirements of unsteady adjoint by synergistically using check‐pointing and compression

Computationally efficient full-waveform inversion of the brain using frequency-adaptive grids and lossy compression

Contact Info

Product

Resources

About