GreeM: Massively Parallel TreePM Code for Large Cosmological <i>N</i>-body Simulations

Ishiyama, Tomoaki; Fukushige, Toshiyuki; Makino, Junichiro

doi:10.1093/pasj/61.6.1319

Cited by 111 publications

(88 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the time integration we used the GreeM code (Ishiyama et al 2009a) for single supercomputer runs and the SUSHI code (Groen et al 2011) for multi-supercomputer runs. The GreeM code is a massively parallel TreePM code based on the parallel TreePM code of Yoshikawa & Fukushige (2005) for large cosmological N-body simulations.…”

Section: Initial Conditions and Numerical Methodsmentioning

confidence: 99%

The Cosmogrid Simulation: Statistical Properties of Small Dark Matter Halos

Ishiyama

Rieder

Makino

et al. 2013

ApJ

Self Cite

100

104

View full text Add to dashboard Cite

We present the results of the "Cosmogrid" cosmological N-body simulation suites based on the concordance LCDM model. The Cosmogrid simulation was performed in a 30 Mpc box with 2048 3 particles. The mass of each particle is 1.28 × 10 5 M , which is sufficient to resolve ultra-faint dwarfs. We found that the halo mass function shows good agreement with the Sheth & Tormen fitting function down to ∼10 7 M . We have analyzed the spherically averaged density profiles of the three most massive halos which are of galaxy group size and contain at least 170 million particles. The slopes of these density profiles become shallower than −1 at the innermost radius. We also find a clear correlation of halo concentration with mass. The mass dependence of the concentration parameter cannot be expressed by a single power law, however a simple model based on the Press-Schechter theory proposed by Navarro et al. gives reasonable agreement with this dependence. The spin parameter does not show a correlation with the halo mass. The probability distribution functions for both concentration and spin are well fitted by the log-normal distribution for halos with the masses larger than ∼10 8 M . The subhalo abundance depends on the halo mass. Galaxy-sized halos have 50% more subhalos than ∼10 11 M halos have.

show abstract

Section: Initial Conditions and Numerical Methodsmentioning

confidence: 99%

The Cosmogrid Simulation: Statistical Properties of Small Dark Matter Halos

Ishiyama

Rieder

Makino

et al. 2013

ApJ

Self Cite

100

104

View full text Add to dashboard Cite

show abstract

“…The detailed explanation of the GreeM code can be found in [24]. In the rest of this section, we describe two novel techniques that significantly improved the performance.…”

Section: Our Parallel Treepm Codementioning

confidence: 99%

“…In this paper, we describe our MPI/OpenMP hybrid TreePM implementation GreeM [24], which is a massively parallel TreePM code based on the implementation of Yoshikawa & Fukushige (2005) [23] for large cosmological N -body simulations, and present performance results of the largest gravitational N -body simulation that has ever been done in the world. We use one trillion dark matter particles to study the nonlinear evolution of the first dark matter structures in the early Universe.…”

Section: Introductionmentioning

confidence: 99%

4.45 Pflops astrophysical N-body simulation on K computer -- The gravitational trillion-body problem

Ishiyama

Nitadori

Makino

2012

2012 International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

Abstract-As an entry for the 2012 Gordon-Bell performance prize, we report performance results of astrophysical N -body simulations of one trillion particles performed on the full system of K computer. This is the first gravitational trillion-body simulation in the world. We describe the scientific motivation, the numerical algorithm, the parallelization strategy, and the performance analysis. Unlike many previous Gordon-Bell prize winners that used the tree algorithm for astrophysical N -body simulations, we used the hybrid TreePM method, for similar level of accuracy in which the short-range force is calculated by the tree algorithm, and the long-range force is solved by the particlemesh algorithm. We developed a highly-tuned gravity kernel for short-range forces, and a novel communication algorithm for long-range forces. The average performance on 24576 and 82944 nodes of K computer are 1.53 and 4.45 Pflops, which correspond to 49% and 42% of the peak speed.

show abstract

“…We used the GreeM cosmological N-body code [12], which in turn relied on MPWide to facilitate the fast message-passing over wide area networks.…”

Section: The Cosmogrid Projectmentioning

confidence: 99%

MPWide: a light-weight library for efficient message passing over wide area networks

Groen

Rieder

Zwart

2013

Journal of Open Research Software

View full text Add to dashboard Cite

Groen et al 2013 MPWide: a light-weight library for efficient message passing over wide area networks. Journal of Open Research Software, 1: e9, DOI: http://dx.doi.org/10.5334/jors.ah We present MPWide, a light weight communication library which allows efficient message passing over a distributed network. MPWide has been designed to connect application running on distributed (super) computing resources, and to maximize the communication performance on wide area networks for those without administrative privileges. It can be used to provide message-passing between application, move files, and make very fast connections in client-server environments. MPWide has already been applied to enable distributed cosmological simulations across up to four supercomputers on two continents, and to couple two different bloodflow simulations to form a multiscale simulation. Journal of open research softwareGroen et al Art. e9, p. 2 of 8 network of supercomputers. The complexity of the underlying supercomputing and network architectures, as well as the high communication performance required for the CosmoGrid project, required us to develop a library that was both highly configurable and trivial to install, regardless of the underlying (super)computing platform.There are a number of tools which have similarities to MPWide. ZeroMQ [1], is a socket library which supports a wide range of platforms. However, compared with MPWide it does have a heavier dependency footprint. Among other things it depends on uuid-dev, a package that requires administrative privileges to install. In addition, there are several performance optimization parameters which can be tweaked with MPWide but not with ZeroMQ. Additionally, the NetIBIS [3] and the PadicoTM [5] tools provide functionalities similar to MPWide, though NetIBIS is written in Java, which is not widely supported on the compute nodes of supercomputers, and PadicoTM requires the presence of a central rendez-vous server. For fast file transfers, alternatives include GridFTP and various closed-source file transfer software solutions. There are also dedicated tools for running MPI applications across clusters [11,14,2] and for coupling applications to form a multiscale simulation (e.g., MUSCLE [4] and the Jungle Computing System [6]). Summary of research using MPWideMPWide has been applied to support several research and technical projects so far. In this section we summarize these projects, the purpose for which MPWide has been used in these projects, and the performance that we obtained using MPWide. The CosmoGrid projectMPWide has been used extensively in the CosmoGrid project, for which it was originally developed. In this project we required a library that enabled fast message passing between supercomputers and which was trivial to install on PCs, clusters, little Endian Cray-XT4 supercomputers and big Endian IBM Power6 supercomputers. In addition, we needed MPWide to deliver solid communication performance over light paths and dedicated 10Gbps networks, even when these networks were not...

show abstract

GreeM: Massively Parallel TreePM Code for Large Cosmological N-body Simulations

Cited by 111 publications

References 35 publications

The Cosmogrid Simulation: Statistical Properties of Small Dark Matter Halos

The Cosmogrid Simulation: Statistical Properties of Small Dark Matter Halos

4.45 Pflops astrophysical N-body simulation on K computer -- The gravitational trillion-body problem

MPWide: a light-weight library for efficient message passing over wide area networks

Contact Info

Product

Resources

About