Reducing Training Time in a One-Shot Machine Learning-Based Compiler

Thomson, John; O’Boyle, Michael; Fursin, Grigori; Franke, Björn

doi:10.1007/978-3-642-13374-9_28

Cited by 14 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Principal Component Analysis (PCA) [3,15,18,19,21,28,29,61,78,142,198,199,202,248] Factor Analysis (FA) [21] 3.2.2 Architecture Independent Characterization. The information collected from a dynamic characterization, referred to as feature vector, is a compact summary of an application's dynamic behavior at run-time.…”

Section: Classi Cation Referencesmentioning

confidence: 99%

A Survey on Compiler Autotuning using Machine Learning

et al. 2018

View full text Add to dashboard Cite

Since the mid-1990s, researchers have been trying to use machine-learning based approaches to solve a number of di erent compiler optimization problems. These techniques primarily enhance the quality of the obtained results and, more importantly, make it feasible to tackle two main compiler optimization problems: optimization selection (choosing which optimizations to apply) and phase-ordering (choosing the order of applying optimizations). The compiler optimization space continues to grow due to the advancement of applications, increasing number of compiler optimizations, and new target architectures. Generic optimization passes in compilers cannot fully leverage newly introduced optimizations and, therefore, cannot keep up with the pace of increasing options. This survey summarizes and classi es the recent advances in using machine learning for the compiler optimization eld, particularly on the two major problems of (1) selecting the best optimizations, and (2) the phase-ordering of optimizations. The survey highlights the approaches taken so far, the obtained results, the ne-grain classi cation among di erent approaches and nally, the in uential papers of the eld. A. H. Ashouri et al.unrolling, register allocation, etc.) could substantially bene t several performance metrics. Depending on the objectives, these metrics could be execution time, code size, or power consumption. A holistic exploration approach to trade-o these metrics also represents a challenging problem [193].Autotuning [35,256] addresses automatic code-generation and optimization by using di erent scenarios and architectures. It constructs techniques for automatic optimization of di erent parameters to maximize or minimize the satis ability of an objective function. Historically, several optimizations were done in the backend where scheduling, resource-allocation and code-generation are done [56,93]. The constraints and resources form a linear system (ILP) which needs to be solved. Recently, researchers have shown increased e ort in introducing front-end and IR-optimizations. Two observations support this claim: (1) the complexity of a backend compiler requires exclusive knowledge strictly by the compiler designers, and (2) lower overheads with external compiler modi cation compared with back-end modi cations. The IR-optimization process normally involves ne-tuning compiler optimization parameters by a multi-objective optimization formulation which can be harder to explore. Nonetheless, each approach has its bene ts and drawbacks and are subject to analysis under their scope.A major challenge in choosing the right set of compiler optimizations is the fact that these code optimizations are programming language, application, and architecture dependent. Additionally, the word optimization is a misnomer -there is no guarantee the transformed code will perform better than the original version. In fact, aggressive optimizations can even degrade the performance of the code to which they are applied [251]. Understanding the behavior of the optimization...

show abstract

Section: Classi Cation Referencesmentioning

confidence: 99%

A Survey on Compiler Autotuning using Machine Learning

et al. 2018

View full text Add to dashboard Cite

show abstract

“…The problem with this approach is the need to build a program-specific model, and the fact that it does not reuse knowledge acquired from previous compilation. Thomson et al [2009] try to reduce the training time by focusing it on the programs that best characterize the search space of the optimization sets. They first gather the static features of all the programs in the training set, then apply unsupervised clustering in the program feature space.…”

Section: Related Workmentioning

confidence: 99%

Continuous learning of compiler heuristics

Tartara

Reghizzi

2013

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Optimizing programs to exploit the underlying hardware architecture is an important task. Much research has been done on enabling compilers to find the best set of code optimizations that can build the fastest and less resource-hungry executable for a given program. A common approach is iterative compilation, sometimes enriched by machine learning techniques. This provides good results, but requires extremely long compilation times and an initial training phase lasting even for days or weeks.We present long-term learning, a new algorithm that allows the compiler user to improve the performance of compiled programs with reduced compilation times with respect to iterative compilation, and without an initial training phase. Our algorithm does not just build good programs: it acquires knowledge every time a program is compiled and it uses such knowledge to learn compiler heuristics, without the need for an expert to manually define them. The heuristics are evolved during every compilation, by evaluating their effect on the generated programs. We present implementations of long-term learning on top of two different compilers, and experimental data gathered on multiple hardware configurations showing its effectiveness.

show abstract

“…PCA has been used in many prior compiler research works for feature reduction [96], [25], [55], [97], [93], [98], [144], [17]. It has also been used in prior works to visualise the working mechanism of a machine learning model, e.g.…”

Section: Feature Selection and Dimension Reductionmentioning

confidence: 99%

Machine Learning in Compiler Optimization

2018

View full text Add to dashboard Cite

Abstract-In the last decade, machine learning based compilation has moved from an an obscure research niche to a mainstream activity. In this article, we describe the relationship between machine learning and compiler optimisation and introduce the main concepts of features, models, training and deployment. We then provide a comprehensive survey and provide a road map for the wide variety of different research areas. We conclude with a discussion on open issues in the area and potential research directions. This paper provides both an accessible introduction to the fast moving area of machine learning based compilation and a detailed bibliography of its main achievements.

show abstract

Reducing Training Time in a One-Shot Machine Learning-Based Compiler

Cited by 14 publications

References 11 publications

A Survey on Compiler Autotuning using Machine Learning

A Survey on Compiler Autotuning using Machine Learning

Continuous learning of compiler heuristics

Machine Learning in Compiler Optimization

Contact Info

Product

Resources

About