No abstract

In the universe’s most massive galaxies, active galactic nucleus (AGN) feedback appears to limit star formation. The accumulation of cold gas near the central black hole fuels powerful AGN outbursts, keeping the ambient medium in a state marginally unstable to condensation and formation of cold gas clouds. However, the ability of that mechanism to self-regulate may depend on numerous environmental factors, including the depth of the potential well and the pressure of the surrounding circumgalactic medium (CGM). Here we present a suite of numerical simulations, with halo mass ranging from 2 × 1012 M ⊙ to 8 × 1014 M ⊙, exploring the dependence of AGN feedback on those environmental factors. We include the spatially extended mass and energy input from the massive galaxy’s old stellar population capable of sweeping gas out of the galaxy if the confining CGM pressure is sufficiently low. Our simulations show that this feedback mechanism is tightly self-regulating in a massive galaxy with a deep central potential and low CGM pressure, permitting only small amounts of multiphase gas to accumulate and allowing no star formation. In a similar-mass galaxy with shallower central potential and greater CGM pressure the feedback mechanism is more episodic, producing extended multiphase gas and allowing small rates of star formation (∼0.1 M ⊙ yr−1). At the low-mass end, the mechanism becomes implausibly explosive, perhaps because the CGM initially has no angular momentum, which would have reduced the amount of condensed gas capable of fueling feedback.

Large scale simulations are a key pillar of modern research and require ever increasing computational resources. Different novel manycore architectures have emerged in recent years on the way towards the exascale era. Performance portability is required to prevent repeated non-trivial refactoring of a code for different architectures. We combine ATHENA++, an existing magnetohydrodynamics (MHD) CPU code, with KOKKOS, a performance portable on-node parallel programming paradigm, into K-ATHENA to allow efficient simulations on multiple architectures using a single codebase. We present profiling and scaling results for different platforms including Intel Skylake CPUs, Intel Xeon Phis, and NVIDIA GPUs. K-ATHENA achieves > 10 8 cell-updates/s on a single V100 GPU for second-order double precision MHD calculations, and a speedup of 30 on up to 24,576 GPUs on Summit (compared to 172,032 CPU cores), reaching 1.94 × 10 12 total cell-updates/s at 76% parallel efficiency. Using a roofline analysis we demonstrate that the overall performance is currently limited by DRAM bandwidth and calculate a performance portability metric of 83.1%. Finally, we present the implementation strategies used and the challenges encountered in maximizing performance. This will provide other research groups with a straightforward approach to prepare their own codes for the exascale era. K-ATHENA is available at https://gitlab.com/pgrete/kathena.

On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multidimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of 1.7 × 1013 zone-cycles/s on 9216 nodes (73,728 logical GPUs) at [Formula: see text] weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.

In cool-core galaxy clusters with central cooling times much shorter than a Hubble time, condensation of the ambient central gas is regulated by a heating mechanism, probably an active galactic nucleus. Previous analytical work has suggested that certain radial distributions of heat input may result in convergence to a quasi-steady global state that does not substantively change on the timescale for radiative cooling, even if the heating and cooling are not locally in balance. To test this hypothesis, we simulate idealized galaxy cluster halos using the ENZO code with an idealized, spherically symmetric heat input kernel intended to emulate. Thermal energy is distributed with radius according to a range of kernels, in which total heating is updated to match total cooling every 10 Myr. Some heating kernels can maintain quasi-steady global configurations, but no kernel we tested produces a quasi-steady state with central entropy as low as those observed in cool-core clusters. The general behavior of the simulations depends on the proportion of heating in the inner 10 kpc, with low central heating leading to central cooling catastrophes, high central heating creating a central convective zone with an inverted entropy gradient, and intermediate central heating resulting in a flat central entropy profile that exceeds observations. The timescale on which our simulated halos fall into an unsteady multiphase state is proportional to the square of the cooling time of the lowest-entropy gas, allowing more centrally concentrated heating to maintain a longer-lasting steady state.