Adaptive Optimization of Sparse Matrix-Vector Multiplication on Emerging Many-Core Architectures

Fang

et al. 2019

Int J Parallel Prog

Self Cite

Sparse matrix-vector multiplications (SpMV) are common in scientific and HPC applications but are hard to be optimized. While the ARMv8based processor IP is emerging as an alternative to the traditional x64 HPC processor design, there is little study on SpMV performance on such new many-cores. To design efficient HPC software and hardware, we need to understand how well SpMV performs. This work develops a quantitative approach to characterize SpMV performance on a recent ARMv8-based many-core architecture, Phytium FT-2000 Plus (FTP). We perform extensive experiments involved over 9,500 distinct profiling runs on 956 sparse datasets and five mainstream sparse matrix storage formats, and compare FTP against the Intel Knights Landing many-core. We experimentally show that picking the optimal sparse matrix storage format and parameters is non-trivial as the correct decision requires expert knowledge of the input matrix and the hardware. We address the problem by proposing a machine learning based model that predicts the best storage format and parameters using input matrix features. The model automatically specializes to the many-core architectures we considered. The experimental results show that our approach achieves on average 93% of the best-available performance without incurring runtime profiling overhead.

Section: Related Workmentioning

confidence: 91%

“…Thus, the choice of K can have an impact of the SpMV performance. In this experiment, we compare two algorithms for choosing K: an average based algorithm [6] and a "histogram" based scheme [2]. This evaluation is performed on four matrices listed in Figure 5(a) to keep the experiments manageable.…”

Section: Hybmentioning

confidence: 99%

Optimizing Sparse Matrix–Vector Multiplications on an ARMv8-based Many-Core Architecture

Fang

et al. 2019

Int J Parallel Prog

Self Cite

“…Shizhao Chen, perform a comprehensive study on representation of sparse matrix on Intel Knights landing XeonPhi and ARM-based FT-200PLUS architecture, they found that best representation of sparse relay on architecture and the unit of program, in their paper they use very well known CSR,CSR5,ELL,SELL and HYB sparse matrix storage format [27].…”

Section: F Compressed Sparse Row-vi(csr-vi)mentioning

confidence: 99%

Evaluate Metadata of Sparse Matrix for SpMV on Shared Memory Architecture

Maruf¹,

Ahmed²

2019

IJACSA

Sparse Matrix operations are frequently used operations in scientific, engineering and high-performance computing (HPC) applications. Among them, sparse matrix-vector multiplication (SpMV) is a popular kernel and considered an important numerical method for science, engineering and in scientific computing. However, SpMV is a computationally expensive operation. To obtain better performance, SpMV depends on certain factors; choosing the right storage format for the sparse matrix is one of them. Other things like data access pattern, the sparsity of the matrix data set, load balancing, sharing of the memory hierarchy, etc. are other factors that affect performance. Metadata, that describes the substructure of the sparse matrix, like shape, density, sparsity, etc. of the sparse matrix also affects performance efficiency for any sparse matrix operation. Various approaches presented in literature over the last few decades given good results for certain types of matrix structures and don't perform as well with others. Developers thus are faced with a difficulty in choosing the most appropriate format. In this research, an approach is presented that evaluates metadata of a given sparse matrix and suggest to the developers the most suitable storage format to use for SpMV.

“…Our adaptive model selection approach allows one to select which model to use based on the input, and is also useful when cloud offloading is prohibitively because of the latency requirement or the lack of connectivity. Machine learning has been employed for various optimization tasks [47,57], including code optimization [6,9,17,37,38,52,[54][55][56][58][59][60][61], task scheduling [10,12,15,16,44], etc. Our approach is closely related to ensemble learning where multiple models are used to solve an optimization problem.…”

Section: Related Workmentioning

confidence: 99%

Adaptive deep learning model selection on embedded systems

et al. 2018

Self Cite

The recent ground-breaking advances in deep learning networks (DNNs) make them attractive for embedded systems.However, it can take a long time for DNNs to make an inference on resource-limited embedded devices. Offloading the computation into the cloud is often infeasible due to privacy concerns, high latency, or the lack of connectivity. As such, there is a critical need to find a way to effectively execute the DNN models locally on the devices. This paper presents an adaptive scheme to determine which DNN model to use for a given input, by considering the desired accuracy and inference time. Our approach employs machine learning to develop a predictive model to quickly select a pre-trained DNN to use for a given input and the optimization constraint. We achieve this by first training off-line a predictive model, and then use the learnt model to select a DNN model to use for new, unseen inputs. We apply our approach to the image classification task and evaluate it on a Jetson TX2 embedded deep learning platform using the ImageNet ILSVRC 2012 validation dataset. We consider a range of influential DNN models. Experimental results show that our approach achieves a 7.52% improvement in inference accuracy, and a 1.8x reduction in inference time over the most-capable single DNN model.