MaSiF: Machine learning guided auto-tuning of parallel skeletons

Collins, Alexander J.; Fensch, Christian; Leather, Hugh; Cole, Murray

doi:10.1109/hipc.2013.6799098

Cited by 7 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to optimizing sequential programs [Cooper et al 1999], recent studies have shown that predictive modeling is effective in optimizing parallel programs Collins et al 2013;Emani et al 2013;Wang et al 2014b] and scheduling parallel workload ]. The Qilin [Luk et al 2009] compiler uses offline profiling to create a regression model that is employed to predict a data parallel program's execution time.…”

Section: Related Workmentioning

confidence: 99%

Automatic and Portable Mapping of Data Parallel Programs to OpenCL for GPU-Based Heterogeneous Systems

Wang

Grewe

O’Boyle

2014

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

General-purpose GPU-based systems are highly attractive, as they give potentially massive performance at little cost. Realizing such potential is challenging due to the complexity of programming. This article presents a compiler-based approach to automatically generate optimized OpenCL code from data parallel OpenMP programs for GPUs. A key feature of our scheme is that it leverages existing transformations, especially data transformations, to improve performance on GPU architectures and uses automatic machine learning to build a predictive model to determine if it is worthwhile running the OpenCL code on the GPU or OpenMP code on the multicore host. We applied our approach to the entire NAS parallel benchmark suite and evaluated it on distinct GPU-based systems. We achieved average (up to) speedups of 4.51× and 4.20× (143× and 67×) on Core i7/NVIDIA GeForce GTX580 and Core i7/AMD Radeon 7970 platforms, respectively, over a sequential baseline. Our approach achieves, on average, greater than 10× speedups over two state-of-the-art automatic GPU code generators.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic and Portable Mapping of Data Parallel Programs to OpenCL for GPU-Based Heterogeneous Systems

Wang

Grewe

O’Boyle

2014

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…For example, varlist my_vars [5] defines a pool named my_vars of size 5. Five variables, named my_vars1 to my_vars5, are in this pool, and can be sampled from using the variable construct.…”

Section: Genesis Flow and Constructsmentioning

confidence: 99%

“…There is increasing interest in the use of machine learning in automatic tuning of program performance, particularly on parallel architectures such as multi-cores and Graphics Processing Units (GPUs) [1,5,8,9,16,19]. Much of this work uses supervised learning methods that rely on training programs, i.e., programs to which various optimizations are applied and whose characteristics and performance are used to build a machine learning model.…”

Section: Introductionmentioning

confidence: 99%

Genesis

Chiu

Garvey

Abdelrahman

2015

Proceedings of the 12th ACM International Conference on Computing Frontiers

View full text Add to dashboard Cite

We describe Genesis, a language for the generation of synthetic programs for use in machine learning-based performance auto-tuning. The language allows users to annotate a template program to customize its code using statistical distributions and to generate program instances based on those distributions. This effectively allows users to generate training programs whose characteristics or features vary in a statistically controlled fashion. We describe the language constructs, a prototype preprocessor for the language, and three case studies that show the ability of Genesis to express a range of training programs in different domains. We evaluate the preprocessor's performance and the statistical quality of the samples it generates. We believe that Genesis is a useful tool for generating large and diverse sets of programs, a necessary component when training machine learning models for auto-tuning.

show abstract

“…In compiler research, the feature sets used for predictive models are often provided without explanation and rarely is the quality of those features evaluated. More commonly, an initial large, high dimensional candidate feature space is pruned via feature selection [3], or projected into a lower dimensional space [43,44]. FEAST employs a range of existing feature selection methods to select useful candidate features [45].…”

Section: Related Workmentioning

confidence: 99%

End-to-End Deep Learning of Optimization Heuristics

Cummins

Petoumenos

Wang

et al. 2017

2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT)

Self Cite

139

154

View full text Add to dashboard Cite

Accurate automatic optimization heuristics are necessary for dealing with the complexity and diversity of modern hardware and software. Machine learning is a proven technique for learning such heuristics, but its success is bound by the quality of the features used. These features must be hand crafted by developers through a combination of expert domain knowledge and trial and error. This makes the quality of the final model directly dependent on the skill and available time of the system architect.Our work introduces a better way for building heuristics. We develop a deep neural network that learns heuristics over raw code, entirely without using code features. The neural network simultaneously constructs appropriate representations of the code and learns how best to optimize, removing the need for manual feature creation. Further, we show that our neural nets can transfer learning from one optimization problem to another, improving the accuracy of new models, without the help of human experts.We compare the effectiveness of our automatically generated heuristics against ones with features hand-picked by experts. We examine two challenging tasks: predicting optimal mapping for heterogeneous parallelism and GPU thread coarsening factors. In 89% of the cases, the quality of our fully automatic heuristics matches or surpasses that of state-of-theart predictive models using hand-crafted features, providing on average 16% and 12% more performance with no human effort expended on designing features.

show abstract

MaSiF: Machine learning guided auto-tuning of parallel skeletons

Cited by 7 publications

References 12 publications

Automatic and Portable Mapping of Data Parallel Programs to OpenCL for GPU-Based Heterogeneous Systems

Automatic and Portable Mapping of Data Parallel Programs to OpenCL for GPU-Based Heterogeneous Systems

Genesis

End-to-End Deep Learning of Optimization Heuristics

Contact Info

Product

Resources

About