Embedded Ensemble Propagation for Improving Performance, Portability, and Scalability of Uncertainty Quantification on Emerging Computational Architectures

Abstract

Quantifying simulation uncertainties is a critical component of rigorous predictive simulation. A key component of this is forward propagation of uncertainties in simulation input data to output quantities of interest. Typical approaches involve repeated sampling of the simulation over the uncertain input data, and can require numerous samples when accurately propagating uncertainties from large numbers of sources. Often simulation processes from sample to sample are similar and much of the data generated from each sample evaluation could be reused. We explore a new method for implementing sampling methods that simultaneously propagates groups of samples together in an embedded fashion, which we call embedded ensemble propagation. We show how this approach takes advantage of properties of modern computer architectures to improve performance by enabling reuse between samples, reducing memory bandwidth requirements, improving memory access patterns, improving opportunities for fine-grained parallelization, and reducing communication costs. We describe a software technique for implementing embedded * ensemble propagation based on the use of C++ templates and describe its integration with various scientific computing libraries within Trilinos. We demonstrate improved performance, portability and scalability for the approach applied to the simulation of partial differential equations on a variety of CPU, GPU, and accelerator architectures, including up to 131,072 cores on a Cray XK7 (Titan).

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

“…Conceptually our array wrapper is similar to SIMD data types studied by others [17,14,31,15,24], which have been used to achieve some form of outer-loop vectorization by blocking the outer loop based on the architecture's vector width and moving the vector loop to an innermost loop encapsulated by overloaded operators iterating over a statically-sized array. Furthermore, these implementations have focused exclusively on vector CPU/Phi architectures with compile-time fixed array lengths.…”

Prediction and reduction of runtime in non-intrusive forward UQ simulations

“…To our knowledge, such an approach has not yet been explored in detail: Only in [21,22] a synthetic output of interest from the computer model is used to predict the number of internal iterations one run takes. With this information, the authors group the runs with similar iterations together to an so called ensemble group.…”

Ensemble Grouping Strategies for Embedded Stochastic Collocation Methods Applied to Anisotropic Diffusion Problems

“…Therefore we have recently undertaken work that further attempts to reduce computational cost for sampling-based UQ methods by reducing the cost of the evaluation of each sample. In particular we have shown [45] that performance can be substantially improved when multiple samples are propagated through a computational simulation together, a technique we call embedded ensemble propagation. In [45] ensembles of samples were propagated through a canonical model of a stochastic isotropic diffusion equation with uncertain diffusion coefficient by replacing all sample-dependent scalars within the simulation code 1 with small arrays.…”

“…In particular we have shown [45] that performance can be substantially improved when multiple samples are propagated through a computational simulation together, a technique we call embedded ensemble propagation. In [45] ensembles of samples were propagated through a canonical model of a stochastic isotropic diffusion equation with uncertain diffusion coefficient by replacing all sample-dependent scalars within the simulation code 1 with small arrays. It was found that the cost of assembling and solving the resulting linear equations of the ensemble system was substantially smaller compared to assembling and solving each system sequentially when implemented on a variety of contemporary and emerging computational architectures for several reasons:…”

“…This paper is organized as follows. We review local hierarchical stochastic collocation methods applied to stochastic PDEs (SPDEs) in section 2 and the ensemble propagation technique from [45] in section 3. Then in section 4 several grouping strategies are discussed, and results of applying these strategies to anisotropic diffusion problems are presented in section 5.…”