Characterizing Scalability of Sparse Matrix–Vector Multiplications on Phytium FT-2000+

Abstract

Understanding the scalability of parallel programs is crucial for software optimization and hardware architecture design. As HPC hardware is moving towards many-core design, it becomes increasingly difficult for a parallel program to make effective use of all available processor cores. This makes scalability analysis increasingly important. This paper presents a quantitative study for characterizing the scalability of sparse matrix-vector multiplications (SpMV) on Phytium FT-2000+, an ARM-based HPC many-core architecture. We choose SpMV as it is a common operation in scientific and HPC applications. Due to the newness of ARM-based many-core architectures, there is little work on understanding the SpMV scalability on such hardware design. To close the gap, we carry out a largescale empirical evaluation involved over 1,000 representative SpMV datasets. We show that, while many computation-intensive SpMV applications contain extensive parallelism, achieving a linear speedup is nontrivial on Phytium FT-2000+. To better understand what software and hardware parameters are most important for determining the scalability of a given SpMV kernel, we develop a performance analytical model based on the regression tree. We show that our model is highly effective in characterizing SpMV scalability, offering useful insights to help application developers for better optimizing SpMV on an emerging HPC architecture.

Power modeling for Phytium FT-2000+/64 multi-core architecture

“…To better explore the accuracy of estimation result, the coefficient of determination 2 and RMSE(root mean square deviation) are calculated, as shown in Table 1. It can be concluded that the model is less effective in predicting an instantaneous power spike.…”

“…Prior work in the area of machine-learning-based systems optimisation provides evidence showing that this could be a viable means to overcome the limits in the hardware implementation [6,7,19,20,27,29,30]. Effort in this direction includes performance modeling for the FT-2000+/64 architecture [2]. We leave this as our future work.…”

“…Donglin Chen et al [2] presented a quantitative study for characterizing the scalability of sparse metrics-vector multiplications (SpMV) on Phytium FT-2000+, and show that while many computation-intensive SpMV applications contain extensive parallelism, achieving a linear speedup is non-trivial on this platform. They developed a performance analytical model based on a regression tree, and showed that their model is highly effective in characterizing SpMV scalability.…”

Performance Evaluation of Memory-Centric ARMv8 Many-Core Architectures: A Case Study with Phytium 2000+

“…On the Phytium 2000+ platform, SELL is the optimal format for around 50% of the sparse matrices and ELL gives the worst performance on most of the cases. Therefore we need an adaptive scheme to help developers to choose the optimal sparse matrix format [20][21][22] .…”

Parallel programming models for heterogeneous many-cores: a comprehensive survey

“…There are many studies showing it outperforms human-based approaches. Recent work shows that it is effective in performing parallel code optimization (Chen et al 2020;Cummins et al 2017a, b;Grewe et al 2013b;Ogilvie et al 2014;Wang et al 2014aWang et al , 2015, performance predicting (Wang and O'Boyle 2013;Zhao et al 2016), parallelism mapping (Grewe et al 2013a;Taylor et al 2017;Tournavitis et al 2009;Wang and O'Boyle 2010;Wang et al 2014bWang et al , 2015Wen et al 2014;Zhang et al 2020), and task scheduling (Emani et al 2013;Marco et al 2017;Ren et al 2017Ren et al , 2018Ren et al , 2020Sanz Marco et al 2019;Yuan et al 2019). As the many-core design becomes increasingly diverse, we believe that the machinelearning techniques provide a rigorous, automatic way for constructing optimization heuristics, which is more scalable and sustainable, compared to manually-crafted solutions.…”