We have performed the first-ever numerical Nbody simulation of the full observable universe (DEUS "Dark Energy Universe Simulation" FUR "Full Universe Run"). This has evolved 550 billion particles on an Adaptive Mesh Refinement grid with more than two trillion computing points along the entire evolutionary history of the universe and across 6 order of magnitudes length scales, from the size of the Milky Way to that of the whole observable Universe. To date, this is the largest and most advanced cosmological simulation ever run. It provides unique information on the formation and evolution of the largest structure in the universe and an exceptional support to future observational programs dedicated to mapping the distribution of matter and galaxies in the universe. The simulation has run on 4752 (of 5040) thin nodes of BULL supercomputer CURIE, using more than 300 TB of memory for 10 million hours of computing time. About 50 PBytes of data were generated throughout the run. Using an advanced and innovative reduction workflow the amount of useful stored data has been reduced to 500 TBytes.
We report the realization of the first cosmological simulations on the scale of the whole observable universe. These simulations have been carried out on 4752 nodes of the Curie supercomputer as a part of the Dark Energy Universe Simulation: Full Universe Runs (DEUS-FUR) project which aims at establishing new probes to put constraints on the nature of dark energy by comparing the growth of large-scale structures, the characteristics of extreme statistical events and the matter distribution in redshift space. The numerical challenge of the first DEUS-FUR simulation associated with the concordance LCDM (L Cold Dark Matter) model was already presented during the 2012 supercomputing conference (Alimi et al., 2012, in The international conference for high performance computing, networking, storage and analysis).Here we first focus on the numerical aspects of the two new simulations. In practice, each one of these simulations has evolved 550 billion dark matter particles in an adaptive mesh refinement grid, and one of the new simulations has pushed back the total number of grid points from 2000 billion for the LCDM model to 2200 billion due to the formation of a larger number of structures. We highlight the optimizations and adjustments required to run such a set of simulations and we then summarize some important lessons learnt for future exascale computing projects.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.