Practical Implementation of Lattice QCD Simulation on Intel Xeon Phi Knights Landing

Abstract

We investigate implementation of lattice Quantum Chromodynamics (QCD) code on the Intel Xeon Phi Knights Landing (KNL). The most time consuming part of the numerical simulations of lattice QCD is a solver of linear equation for a large sparse matrix that represents the strong interaction among quarks. To establish widely applicable prescriptions, we examine rather general methods for the SIMD architecture of KNL, such as using intrinsics and manual prefetching, to the matrix multiplication and iterative solver algorithms. Based on the performance measured on the Oakforest-PACS system, we discuss the performance tuning on KNL as well as the code design for facilitating such tuning on SIMD architecture and massively parallel machines.

Practical Implementation of Lattice QCD Simulation on SIMD Machines with Intel AVX-512

“…An approach keeping the array of structure data layout and inserting pragmas [9] gives 225 GFlops (245 GFlops after correcting the difference in clock cycle). In our previous report [3], which corresponds to the layout 2 without redundant boundary data packing/copy, the best performance on single node was 340 GFlops (4 MPI proc./node). With the same condition, it becomes 369 GFlops whose improvement is mainly due to the refinement of the prefetch.…”

“…As a testbed of our analysis, we choose two types of fermion matrices together with an iterative linear equation solver. In our previous report [2,3], we developed a code along the above policy and applied it to KNL. In this paper, in addition to improved performance, we rearrange these prescriptions so that each effect is more apparent.…”

A survey on evaluating and optimizing performance of Intel Xeon Phi

“…Data layouts SoA, 20,22,23,28,30,36,50,79,82,90 AoS, 9 AoSoA 57,90 Data alignment 6,9,14,18,20,24,44,45,52,53,66,79,84,90 Padding 4,7,9,20,24,44,52,53,79,82,91 Dependency disambiguation 15,28,36,82,91 Prefetching Software, 4,7,9,14,17,22,23,40,41,50,…”

“…Kanamori et al 66 accelerate "lattice quantum chromodynamics" (QCD) code on KNL. For the complex vector data, the real and imaginary parts are placed consecutively in the memory.…”

“…Vectorization-related Difficulties in or imperfect vectorization, 2,22,25 incompatibility of SSE-style vectorization 2 Poor scalability with increasing threads/nodes 2, 16,20,29,37,57,62,66,84 Slow operations reduction operation, 86 gather-scatter operations, 82 divide operations 13,26 Data dependency 99 Load-imbalance between threads of Phi, 13,17,23,50 CPU and Phi 18,43,50,76 Contention on shared components ring stop, 29,37,84 disk-access 37 Others lack of support for vector atomic instructions, 37 thread-creation overhead, 30 TLB misses 42,90 intrinsic functions for introducing vector variables and processes. This approach provides better performance compared to the compiler performing auto-vectorization.…”