Autotuning Runtime Specialization for Sparse Matrix-Vector Multiplication

Abstract

Runtime specialization is used for optimizing programs based on partial information available only at runtime. In this paper we apply autotuning on runtime specialization of Sparse Matrix-Vector Multiplication to predict a best specialization method among several. In 91% to 96% of the predictions, either the best or the second-best method is chosen. Predictions achieve average speedups that are very close to the speedups achievable when only the best methods are used. By using an efficient code generator and a carefully designed set of matrix features, we show the runtime costs can be amortized to bring performance benefits for many real-world cases.

A sparse matrix‐vector multiplication method with low preprocessing cost

“…The categorization of the matrix is to be made based on install‐time information learned on the target machine and a dynamic analysis of the matrix. This is a typical auto‐tuning approach that has been successfully applied in the case of SpMV . Ideally, the dynamic analysis of the matrix should be as inexpensive as possible.…”

“…This is a typical auto-tuning approach that has been successfully applied in the case of SpMV. [15][16][17][18][19][20] Ideally, the dynamic analysis of the matrix should be as inexpensive as possible. In the case of CSRLenGoto, one can quickly check whether the given matrix falls into the category of "short rows" or "long rows".…”

“…The relative cost of preprocessing tends to be higher for matrices with short rows compared to the matrices with long rows. Yet, the costs are very small, considering that in an expensive SpMV method, preprocessing may take time equivalent to tens or hundreds of SpMV invocations . On the average, CSRLen preprocessing time is equivalent to 0.25 SpMV executions on Intel and 0.11 on AMD.…”

“…Reordering the matrix elements and using a storage format other than CSR may give substantial speedup for some matrices . However, the cost of preprocessing and storage format conversion in these approach can be equivalent to tens or even hundreds of SpMV iterations . Compensating these costs can be possible only if the optimized SpMV routine will be called hundreds or thousands of times.…”

SCALE-EA: A Scalability Aware Performance Tuning Framework for OpenMP Applications

“…Studies have also led a subset of researchers to frame autotuning solutions at compile time [2,9,12,27,32] and a few others at run time of applications [37,3,40,8]. In addition, a few autotuning solutions with more emphasis to IOs were designed in [5].…”