Scalable Earthquake Simulation on Petascale Supercomputers

Cui, Yifeng; Olsen, Kim B.; Jordan, Thomas H.; Lee, Kwangyoon; Zhou, Jun; Small, Patrick; Roten, Daniel; Ely, G.; Panda, Dhabaleswar K.; Chourasia, Amit; Levesque, John; Day, Steven M.; Maechling, P. J.

doi:10.1109/sc.2010.45

Cited by 124 publications

(75 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latest tool for assimilating this information into seismic velocity models is fully three-dimensional (3-D) waveform tomography [e.g., Tarantola, 1988;Tromp et al, 2005;Chen et al, 2007aChen et al, , 2007bBen Hadj Ali et al, 2009a, 2009bChen, 2011;Fichtner, 2011;Lekic and Romanowicz, 2011;Liu and Gu, 2012;Colli et al, 2013;Fichtner et al, 2013;French et al, 2013;Prieux et al, 2013;Schiemenz and Igel, 2013]. Full-3-D tomography (F3DT) accounts for the physics of wave excitation and propagation by numerically solving the inhomogeneous equations of motion for a heterogeneous, anelastic solid [Olsen et al, 1995;Komatitsch and Tromp, 1999;Cui et al, 2010;Peter et al, 2011]. The starting model is 3-D, and the Fréchet (sensitivity) kernels are calculated using the full physics of 3-D wave propagation [Zhao et al, 2005[Zhao et al, , 2006Tromp et al, 2005;Liu and Tromp, 2006].…”

Section: Introductionmentioning

confidence: 99%

Full‐3‐D tomography for crustal structure in Southern California based on the scattering‐integral and the adjoint‐wavefield methods

et al. 2014

Self Cite

View full text Add to dashboard Cite

We have successfully applied full-3-D tomography (F3DT) based on a combination of the scattering-integral method (SI-F3DT) and the adjoint-wavefield method (AW-F3DT) to iteratively improve a 3-D starting model, the Southern California Earthquake Center (SCEC) Community Velocity Model version 4.0 (CVM-S4). In F3DT, the sensitivity (Fréchet) kernels are computed using numerical solutions of the 3-D elastodynamic equation and the nonlinearity of the structural inversion problem is accounted for through an iterative tomographic navigation process. More than half-a-million misfit measurements made on about 38,000 earthquake seismograms and 12,000 ambient-noise correlagrams have been assimilated into our inversion. After 26 F3DT iterations, synthetic seismograms computed using our latest model, CVM-S4.26, show substantially better fit to observed seismograms at frequencies below 0.2 Hz than those computed using our 3-D starting model CVM-S4 and the other SCEC CVM, CVM-H11.9, which was improved through 16 iterations of AW-F3DT. CVM-S4.26 has revealed strong crustal heterogeneities throughout Southern California, some of which are completely missing in CVM-S4 and CVM-H11.9 but exist in models obtained from previous crustal-scale 2-D active-source refraction tomography models. At shallow depths, our model shows strong correlation with sedimentary basins and reveals velocity contrasts across major mapped strike-slip and dip-slip faults. At middle to lower crustal depths, structural features in our model may provide new insights into regional tectonics. When combined with physics-based seismic hazard analysis tools, we expect our model to provide more accurate estimates of seismic hazards in Southern California.

show abstract

Section: Introductionmentioning

confidence: 99%

Full‐3‐D tomography for crustal structure in Southern California based on the scattering‐integral and the adjoint‐wavefield methods

et al. 2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…Cui et al [5] demonstrated respective earthquake and seismic hazard simulations, recently also utilizing heterogeneous platforms: for simulations on the GPU-accelerated Titan supercomputer, they reported a speed-up of 5 due to using GPUs [4]. That the reported performance is less than 5% of the theoretical peak performance of the GPU accelerators shows the challenge of optimizing production codes for these platforms.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Sustained Petascale Performance of Seismic Simulations with SeisSol on SuperMUC

Breuer

Heinecke

Rettenberger

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Seismic simulations in realistic 3D Earth models require petaor even exascale computing power to capture small-scale features of high relevance for scientific and industrial applications. In this paper, we present optimizations of SeisSol -a seismic wave propagation solver based on the Arbitrary high-order accurate DERivative Discontinuous Galerkin (ADER-DG) method on fully adaptive, unstructured tetrahedral meshes -to run simulations under production conditions at petascale performance. Improvements cover the entire simulation chain: from an enhanced ADER time integration via highly scalable routines for mesh input up to hardware-aware optimization of the innermost sparse-/densematrix kernels. Strong and weak scaling studies on the SuperMUC machine demonstrated up to 90% parallel efficiency and 45% floating point peak efficiency on 147k cores. For a simulation under production conditions (10 8 grid cells, 5 · 10 10 degrees of freedom, 5 seconds simulated time), we achieved a sustained performance of 1.09 PFLOPS.

show abstract

“…ADIOS implements a variety of techniques to improve I/O performance, and many applications that have encountered these problems in production runs now use ADIOS, e.g., S3D [1], XGC [15] and M8 earthquake simulation [16]. For example, ADIOS enables applications to configure their output buffer size M .…”

Section: Introductionmentioning

confidence: 99%

Characterizing output bottlenecks in a supercomputer

Xie

Chase

Dillow

et al. 2012

2012 International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Abstract-Supercomputer I/O loads are often dominated by writes. HPC (High Performance Computing) file systems are designed to absorb these bursty outputs at high bandwidth through massive parallelism. However, the delivered write bandwidth often falls well below the peak. This paper characterizes the data absorption behavior of a center-wide shared Lustre parallel file system on the Jaguar supercomputer. We use a statistical methodology to address the challenges of accurately measuring a shared machine under production load and to obtain the distribution of bandwidth across samples of compute nodes, storage targets, and time intervals. We observe and quantify limitations from competing traffic, contention on storage servers and I/O routers, concurrency limitations in the client compute node operating systems, and the impact of variance (stragglers) on coupled output such as striping. We then examine the implications of our results for application performance and the design of I/O middleware systems on shared supercomputers.

show abstract

Scalable Earthquake Simulation on Petascale Supercomputers

Cited by 124 publications

References 32 publications

Full‐3‐D tomography for crustal structure in Southern California based on the scattering‐integral and the adjoint‐wavefield methods

Full‐3‐D tomography for crustal structure in Southern California based on the scattering‐integral and the adjoint‐wavefield methods

Sustained Petascale Performance of Seismic Simulations with SeisSol on SuperMUC

Characterizing output bottlenecks in a supercomputer

Contact Info

Product

Resources

About