MILC Staggered Conjugate Gradient Performance on Intel KNL

Abstract

We review our work done to optimize the staggered conjugate gradient (CG) algorithm in the MILC code for use with the Intel Knights Landing (KNL) architecture. KNL is the second generation Intel Xeon Phi processor. It is capable of massive thread parallelism, data parallelism, and high on-board memory bandwidth and is being adopted in supercomputing centers for scientific research. The CG solver consumes the majority of time in production running, so we have spent most of our effort on it. We compare performance of an MPI+OpenMP baseline version of the MILC code with a version incorporating the QPhiX staggered CG solver, for both one-node and multi-node runs.34th annual International Symposium on Lattice Field Theory

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

“…Moreover, both works analyze traditional scientific applications while we also evaluate the data-analytics applications. Other studies [5], [6] focus on the performance of a specific application on a KNL system and show how to optimize the application to increase the performance benefit. More specifically, Bálint et al [6] shows of a performance analysis of linear solvers and kernels used in Lattice Quantumchromodynamics (QCD) on 16 KNL nodes.…”

“…This work also emphasizes the importance of using hyper-threads to effectively exploit HBM bandwidth. DeTar et al [5] optimize a staggered conjugate gradient solver in the MILC application for KNL and report that use of HBM brings a large improvement for the MILC application.…”

“…Thus it is important to understand how applications can make efficient use of these new high-bandwidth memory technologies and what the limitations and constraints are [4]. While several studies have successfully ported specific applications to hardware accelerators equipped with 3D-stacked memory [5], [6], a deep and comprehensive understanding on how to leverage the high-bandwidth memory in HPC applications is still an open issue. In this paper we target this problem and aim at providing recommendations to programmers on which applications may benefit from these new memory technologies and what are the limiting factors to consider and their impact on the application performance.…”

Status and future perspectives for lattice gauge theory calculations to the exascale and beyond

“…Overall three USQCD codes (Chroma, MILC, CPS) askwere chosen to be Tier-1 NERSC Exascale Application Partnership (NESAP) codes, with another (QLua) becoming a Tier-2 code. As part of this partnership, these codes were further developed and optimized through hackathons and dungeon sessions with NERSC and Intel [106,107]. In the context of USQCD, Jefferson Lab deployed a KNL cluster in 2016 and enlarged it in 2018.…”

Three Dirac operators on two architectures with one piece of code and no hassle

“…From a HPC viewpoint, a clear advantage of this operator with precomputed V µ is that its stencil is restricted to sites which are at most one hop away. Still, it is not trivial to reach an acceptable performance on a many-core architecture [4,5].…”