A 1024-Member Ensemble Data Assimilation with 3.5-Km Mesh Global Weather Simulations

Yashiro, Hisashi; Terasaki, Koji; Kawai, Yuka; Kudo, Shuhei; Miyoshi, Takemasa; Imamura, Toshiyuki; Minami, Kazuo; Inoue, Hikaru; Nishiki, Tatsuo; Saji, Takayuki; Satoh, Masaki; Tomita, Hirofumi

doi:10.1109/sc41405.2020.00005

Cited by 24 publications

(20 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The runners apply the numerical climate model to propagate the analysis states to the next observation timestep. Each runner may need very large allocations comprising several nodes, depending on the complexity of the model (for instance 512 nodes for the NICAM atmospheric model [28]). In order to interface with the Melissa-DA framework, the simulation model needs to implement two API functions: (1) melissa_init and (2) melissa_expose.…”

Section: Runnermentioning

confidence: 99%

“…Yet, the degree of resolution is nowhere near saturation. Terasaki, Miyoshi et al have performed studies in 2015, using about 5,700 nodes of the K-Computer at RIKEN reaching 720 teraFLOPS [20], and in 2020 on the Fugaki supercomputer on more than 130,000 nodes reaching 79 petaFLOPS [28]. The amount of memory needed for such simulations already is in the Petabyte regime.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Zero-Waste Recovery and Zero-Overhead Checkpointing in Ensemble Data Assimilation

Keller

Kestelman

Bautista-Gomez

2021

2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)

View full text Add to dashboard Cite

Ensemble data assimilation is a powerful tool for increasing the accuracy of climatological states. It is based on combining observations with the results from numerical model simulations. The method comprises two steps, (1) the propagation, where the ensemble states are advanced by the numerical model and (2) the analysis, where the model states are corrected with observations. One bottleneck in ensemble data assimilation is circulating the ensemble states between the two steps. Often, the states are circulated using files. This article presents an extended implementation of Melissa-DA, an in-memory ensemble data assimilation framework, allowing zero-overhead checkpointing and recovery with few or zero recomputation. We hide the checkpoint creation using dedicated threads and MPI processes. We benchmark our implementation with up to 512 members simulating the Lorenz96 model using 10 9 gridpoints. We utilize up to 8 K processes and 8 TB of checkpoint data per cycle and reach a peak performance of 52 teraFLOPS.

show abstract

Section: Runnermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Towards Zero-Waste Recovery and Zero-Overhead Checkpointing in Ensemble Data Assimilation

Keller

Kestelman

Bautista-Gomez

2021

2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)

View full text Add to dashboard Cite

show abstract

“…This study aims to investigate the predictability and the causes of the heavy rainfall event by performing 1,024-member data assimilation and forecast experiments using NICAM-LETKF ( (Terasaki et al 2015) and satellite radiances (Terasaki and Miyoshi 2017). It has been optimized for Fugaku to perform experiments at high resolution and with large ensemble sizes (Yashiro et al 2016, Yashiro et al 2020.…”

Section: Introductionmentioning

confidence: 99%

“…Finally, conclusions are given in section 5. 5 The 1,024 ensemble initial conditions are the same as those of Yashiro et al (2020).…”

Section: Introductionmentioning

confidence: 99%

A 1024-Member NICAM-LETKF Experiment for the July 2020 Heavy Rainfall Event

Terasaki

Miyoshi

2022

SOLA

Self Cite

View full text Add to dashboard Cite

This study investigated the predictability and causes of the heavy rainfall event that brought severe disasters in Kyushu in July 2020 with a global numerical weather prediction system composed of the NICAM (non-hydrostatic icosahedral atmospheric model) and the LETKF (local ensemble transform Kalman filter). We performed ensemble data assimilation and forecast experiments using the NICAM-LETKF system with 1,024 members and 56-km horizontal resolution on the supercomputer Fugaku.The results showed that 1,024-member ensemble forecasts captured the probability of Terasaki et al., Probability of heavy rainfall with 1024-member experiment 2 heavy rainfall in Kyushu about five days before it happens, although a 10-day-lead forecast is difficult. Ensemble-based lag-correlation analyses with the 1024-member ensemble showed very small sampling errors in the correlation patterns and showed that the moist air inflow in the lower troposphere associated with a low-pressure anomaly over the Baiu front was related to this heavy rainfall in Kyushu.

show abstract

“…For example, important scientific applications such as climate simulations have evolved into multi-million lines of code comprising many algorithms (and their respective kernels) which either execute in parallel or sequentially (see coupled climate models 2 ) depending on availability of computational resources and depending on the inherent load imbalance of the underlying problem (e.g., the location of clouds in the simulated area). Additionally, these workloads may include extensive pre-/post-processing of the data streams or they may assimilate new data from external sensors on the fly 3 . The availability of Noisy Intermediate-Scale Quantum (NISQ) computers, FPGAs and ASICs, in-memory processing, SmartNICs, and the commercial success of Deep Learning and crypto-currencies, increases the choice for processors beyond just traditional CPUs and GPGPUs (see Cambrian explosion in processor architecture 4 ).…”

Section: Introductionmentioning

confidence: 99%

Preparing for the Future -- Rethinking Proxy Apps

Matsuoka¹,

Domke²,

Wahib³

et al. 2022

Preprint

View full text Add to dashboard Cite

A considerable amount of research and engineering went into designing proxy applications, which represent common high-performance computing workloads, to co-design and evaluate the current generation of supercomputers, e.g., RIKEN's Supercomputer Fugaku, ANL's Aurora, or ORNL's Frontier. This process was necessary to standardize the procurement while avoiding duplicated effort at each HPC center to develop their own benchmarks. Unfortunately, proxy applications force HPC centers and providers (vendors) into a an undesirable state of rigidity, in contrast to the fast-moving trends of current technology and future heterogeneity. To accommodate an extremelyheterogeneous future, we have to reconsider how to co-design supercomputers during the next decade, and avoid repeating the past mistakes. This position paper outlines the current state-of-the-art in system co-design, challenges encountered over the past years, and a proposed plan to move forward.

show abstract

A 1024-Member Ensemble Data Assimilation with 3.5-Km Mesh Global Weather Simulations

Cited by 24 publications

References 33 publications

Towards Zero-Waste Recovery and Zero-Overhead Checkpointing in Ensemble Data Assimilation

Towards Zero-Waste Recovery and Zero-Overhead Checkpointing in Ensemble Data Assimilation

A 1024-Member NICAM-LETKF Experiment for the July 2020 Heavy Rainfall Event

Preparing for the Future -- Rethinking Proxy Apps

Contact Info

Product

Resources

About