Collective Mind: Towards Practical and Collaborative Auto-Tuning

Abstract

Abstract. Empirical auto-tuning and machine learning techniques have been showing high potential to improve execution time, power consumption, code size, reliability and other important metrics of various applications for more than two decades. However, they are still far from widespread production use due to lack of native support for auto-tuning in an ever changing and complex software and hardware stack, large and multi-dimensional optimization spaces, excessively long exploration times, and lack of unified mechanisms for preserving and sharing of optimization knowledge and research material.We present a possible collaborative approach to solve above problems using Collective Mind knowledge management system. In contrast with previous cTuning framework, this modular infrastructure allows to preserve and share through the Internet the whole auto-tuning setups with all related artifacts and their software and hardware dependencies besides just performance data. It also allows to gradually structure, systematize and describe all available research material including tools, benchmarks, data sets, search strategies and machine learning models. Researchers can take advantage of shared components and data with extensible meta-description to quickly and collaboratively validate and improve existing auto-tuning and benchmarking techniques or prototype new ones. The community can now gradually learn and improve complex behavior of all existing computer systems while exposing behavior anomalies or model mispredictions to an interdisciplinary community in a reproducible way for further analysis. We present several practical, collaborative and model-driven auto-tuning scenarios. We also decided to release all material at c-mind.org/repo to set up an example for a collaborative and reproducible research as well as our new publication model in computer engineering where experimental results are continuously shared and validated by the community.

Variability of data dependences and control flow

“…Among the contributions of this paper are: 1) We develop a simple, yet powerful profiling based analysis to capture data and control flow dependences for program executions with different input data sets, 2) we analyze the variability of both data and control flow dependences for the whole CBENCH benchmark suite [18], [19] using 100 randomly chosen input data sets from the KDATASETS [20] collection, and 3) we analyze the performance implications of the dynamically collected dependence information with respect to the ability to exploit loop-level parallelism and compare against static parallelization approaches.…”

“…We use the MIDATASETS/CBENCH benchmark suite [19], [22] for our evaluation. It contains 32 benchmarks from the MIBENCH [18] suite.…”

“…For this we apply profile-guided data and control flow dependence analyses across the MIBENCH-derived CBENCH benchmark suite [18], [19] with 100 randomly chosen input data sets from the KDATASETS [20] collection for each benchmark. For each benchmark we analyze and characterize how different input sets cause variation in data and control flow patterns and how these variations in turn affect loop parallelization.…”

Guide Automatic Vectorization by means of Machine Learning: A Case Study of Tensor Contraction Kernels

“…Finally, G. Fursin et al [7] have proposed the notion of iterative compilation; they get rid of software characteristics altogether, and consider (among other) the performance of a program for a given set of optimization strategies in order to predict for the same program. While robust, this approach is obviously only relevant for large optimization spaces, which is not the case in our study.…”

Continuous learning of compiler heuristics

“…Whenever a new program needs to be compiled, the model is queried to predict good configurations. Usually, this just aims at focusing the selection of the configurations to be tested through iterative compilation on more promising areas [Agakov et al 2006], but a similar approach can be used to predict a single configuration to be used as the result of the compilation [Fursin and Temam 2010]. Unfortunately, the training phase for building a good model is really long, up to several weeks [Fursin et al 2008], and this limits the applicability of machine learning approaches.…”