Collective optimization

Fursin, Grigori; Temam, Olivier

doi:10.1145/1880043.1880047

Cited by 37 publications

(25 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the future, compiler independent ICI should help transfer Milepost technology to other compilers. We connected Milepost GCC to a public collective optimization database at cTuning.org [3,38,42]. This provides a wealth of continuously updated training data from multiple users and environments.…”

Section: Introductionmentioning

confidence: 99%

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Fursin

Kashnikov

Memon

et al. 2011

Int J Parallel Prog

Self Cite

216

251

View full text Add to dashboard Cite

Tuning compiler optimizations for rapidly evolving hardware makes porting and extending an optimizing compiler for each new platform extremely challenging. Iterative optimization is a popular approach to adapting programs to a new architecture automatically using feedback-directed compilation. However, the large number of evaluations required for each program has prevented iterative compilation from widespread take-up in production compilers. Machine learning has been proposed to tune optimizations across programs systematically but is currently limited to a few transformations, long training phases and critically lacks publicly released, stable tools.Our approach is to develop a modular, extensible, self-tuning optimization infrastructure to automatically learn the best optimizations across multiple programs and architectures based on the correlation between program features, run-time behavior and optimizations. In this paper we describe Milepost GCC, the first publicly-available open-source machine learning-based compiler. It consists of an Interactive Compilation Interface (ICI) and plugins to extract program features and exchange optimization data with the cTuning.org open public repository. It automatically adapts the internal optimization heuristic at function-level granularity to improve execution time, code size and compilation time of a new program on a given architecture. Part of the Milepost technology together with low-level ICI-inspired plugin framework is now included in the mainline GCC.We developed machine learning plugins based on probabilistic and transductive approaches to predict good combinations of optimizations. Our preliminary experimental results show that it is possible to automatically reduce the execution time of individual MiBench programs, some by more than a factor of 2, while also improving compilation 1 INRIA Saclay, France (HiPEAC member) · 2 University of Versailles Saint Quentin en Yvelines, France · 3 IBM Haifa, Israel (HiPEAC member) · 4 CAPS Entreprise, France (HiPEAC member) · 5 ARC International, UK · 6 University of Edinburgh, UK (HiPEAC member) · 2 time and code size. On average we are able to reduce the execution time of the MiBench benchmark suite by 11% for the ARC reconfigurable processor. We also present a realistic multi-objective optimization scenario for Berkeley DB library using Milepost GCC and improve execution time by approximately 17%, while reducing compilation time and code size by 12% and 7% respectively on Intel Xeon processor.

show abstract

Section: Introductionmentioning

confidence: 99%

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Fursin

Kashnikov

Memon

et al. 2011

Int J Parallel Prog

Self Cite

216

251

View full text Add to dashboard Cite

show abstract

“…However, these representations are less expressive and are often out-performed by more expensive dynamic techniques. Other researchers have proposed using dynamic characterizations of programs; however, techniques (e.g., performance counters [10] and reactions [15,27]) are expensive and require running the program, which limits their practical use.…”

Section: Characterizing the Programmentioning

confidence: 99%

“…Another method of dynamically characterizing a program is through reactions. Here, we record the performance impact from a fixed set of different optimization sequences [9,15]. Thus, we run a program K number of times by optimizing it with K different optimization sequences, and then obtain different speedups for each sequence.…”

Section: Different Ways To Characterize the Programmentioning

confidence: 99%

“…Using a well-constructed machine learning model to choose optimizations for a specific program has repeatedly been shown to outperform the most aggressive optimization levels in open-source and commercial compilers [6,10,13,15,16,19,20,22,25,30]. However, to use machine learning effectively, it is critical to use expressive features that characterize programs well and that strongly correlate to beneficial optimization sequences for the target program.…”

Section: Introductionmentioning

confidence: 99%

“…Methods to dynamically characterize a program have consisted of either using performance counters or a technique known as "reactions." Performance counters can be used to collect information, such as cache behavior, functional unit usage, and dynamic instruction mix from the running program [10,15,22]. Reactions is a technique to dynamically characterize a program by selecting specific program transformations to apply to a program and using the resulting speedups to characterize a program's behavior [9,27].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using graph-based program characterization for predictive modeling

Park

Cavazos

Álvarez

2012

Proceedings of the Tenth International Symposium on Code Generation and Optimization

View full text Add to dashboard Cite

Using machine learning has proven effective at choosing the right set of optimizations for a particular program. For machine learning techniques to be most effective, compiler writers have to develop expressive means of characterizing the program being optimized. The current state-of-the-art techniques for characterizing programs include using a fixed-length feature vector of either source code features extracted during compile time or performance counters collected when running the program. For the problem of identifying optimizations to apply, models constructed using performance counter characterizations of a program have been shown to outperform models constructed using source code features. However, collecting performance counters requires running the program multiple times, and this "dynamic" method of characterizing programs can be specific to inputs of the program. It would be preferable to have a method of characterizing programs that is as expressive as performance counter features, but that is "static" like source code features and therefore does not require running the program.In this paper, we introduce a novel way of characterizing programs using a graph-based characterization, which uses the program's intermediate representation and an adapted learning algorithm to predict good optimization sequences. To evaluate different characterization techniques, we focus on loop-intensive programs and construct prediction models that drive polyhedral optimizations, such as auto-parallelism and various loop transformation.We show that our graph-based characterization technique outperforms three current state-of-the-art characterization techniques found in the literature. By using the sequences predicted to be the best by our graph-based model, we achieved up to 73% of the speedup achievable in our search space for a particular platform, whereas we could only achieve up to 59% by other state-of-the-art techniques we evaluated.

show abstract

On the Determination of Inlining Vectors for Program Optimization

Cammarota

Nicolau

Veidenbaum

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper we propose a new technique and a framework to select inlining heuristic constraints -referred to as an inlining vector, for program optimization. The proposed technique uses machine learning to model the correspondence between inlining vectors and performance (completion time). The automatic selection of a machine learning algorithm to build such a model is part of our technique and we present a rigorous selection procedure. Subject to a given architecture, such a model evaluates the benefit of inlining combined with other global optimizations and selects an inlining vector that, in the limits of the model, minimizes the completion time of a program.We conducted our experiments using the GNU GCC compiler and optimized 22 combinations (program, input) from SPEC CINT2006 on the state-of-the-art Intel Xeon Westmere architecture. Compared with optimization level, i.e., -O3, our technique yields performance improvements ranging from 2% to 9%.

show abstract

Collective optimization

Cited by 37 publications

References 34 publications

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Using graph-based program characterization for predictive modeling

On the Determination of Inlining Vectors for Program Optimization

Contact Info

Product

Resources

About