Parallel Parameter Tuning for Applications with Performance Variability

Tabatabaee, Vahid; Tiwari, Ananta; Hollingsworth, Jeffrey K.

doi:10.1109/sc.2005.52

Cited by 30 publications

(26 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical Search Using DL/ML Model. This section demonstrates the integration of the analytical bounds with existing search optimization algorithms, the Nelder-Mead Simplex method [22] and the Parallel Rank Ordering (PRO) method [30]. In order to handle boundary constraints due to the DL/ML model, we used the extended version of the PRO algorithm introduced in the Active Harmony framework [32].…”

Section: Search Space Reduction By Dl/ml Modelmentioning

confidence: 99%

Analytical Bounds for Optimal Tile Size Selection

Shirako

Sharma

Fauzia

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper, we introduce a novel approach to guide tile size selection by employing analytical models to limit empirical search within a subspace of the full search space. Two analytical models are used together: 1) an existing conservative model, based on the data footprint of a tile, which ignores intra-tile cache block replacement, and 2) an aggressive new model that assumes optimal cache block replacement within a tile. Experimental results on multiple platforms demonstrate the practical effectiveness of the approach by reducing the search space for the optimal tile size by 1,307× to 11,879× for an Intel Core-2-Quad system; 358× to 1,978× for an Intel Nehalem system; and 45× to 1,142× for an IBM Power7 system. The execution of rectangularly tiled code tuned by a search of the subspace identified by our model achieves speed-ups of up to 1.40× (Intel Core-2 Quad), 1.28× (Nehalem) and 1.19× (Power 7) relative to the best possible square tile sizes on these different processor architectures. We also demonstrate the integration of the analytical bounds with existing search optimization algorithms. Our approach not only reduces the total search time from Nelder-Mead Simplex and Parallel Rank Ordering methods by factors of up to 4.95× and 4.33×, respectively, but also finds better tile sizes that yield higher performance in tuned tiled code.

show abstract

Section: Search Space Reduction By Dl/ml Modelmentioning

confidence: 99%

Analytical Bounds for Optimal Tile Size Selection

Shirako

Sharma

Fauzia

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The algorithm that we use is based on the Parallel Rank Order (PRO) algorithm proposed by Tabatabaee et al [22]. For a function of N variables, PRO maintains a set of K (where K is at least N + 1 and is usually set to the number of cores the harmonized application is run on) points forming the vertices of a simplex in an N -dimensional space.…”

Section: Parameter Tuning Algorithmmentioning

confidence: 99%

“…the performance of two consecutive timesteps is recorded) and the minimum of the two samples is sent to the Active Harmony server. We showed in our previous work [22] that even in the presence of 5% variability due to background noise, taking the minimum of two samples is enough to ensure the convergence of the search algorithm.…”

mentioning

confidence: 99%

Online Adaptive Code Generation and Tuning

Tiwari

Hollingsworth

2011

2011 IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

Abstract-In this paper, we present a runtime compilation and tuning framework for parallel programs. We extend our prior work on our auto-tuner, Active Harmony, for tunable parameters that require code generation (for example, different unroll factors). For such parameters, our auto-tuner generates and compiles new code on-the-fly. Effectively, we merge traditional feedback directed optimization and just-in-time compilation. We show that our system can leverage available parallelism in today's HPC platforms by evaluating different code-variants on different nodes simultaneously. We evaluate our system on two parallel applications and show that our system can improve runtime execution by up to 46% compared to the original version of the program.

show abstract

“…Average is the most widely used operator to aggregate and estimate "real" performance from multiple samples. As an alternative, we showed, in our earlier work [83], taking a minimum of multiple performance measurements is an effective way for performance estimation even in the presence of heavy-tail component in the performance distribution. The minimum has finite mean and variance and is not heavy-tailed.…”

Section: Case Study: Gs2mentioning

confidence: 99%

Tuning parallel applications in parallel

2009

View full text Add to dashboard Cite

Auto-tuning has recently received significant attention from the High Performance Computing community. Most auto-tuning approaches are specialized to work either on specific domains such as dense linear algebra and stencil computations, or only at certain stages of program execution such as compile time and runtime.Real scientific applications, however, demand a cohesive environment that can efficiently provide auto-tuning solutions at all stages of application development and deployment. Towards that end, we describe a unified end-to-end approach to autotuning scientific applications. Our system, Active Harmony, takes a search-based collaborative approach to auto-tuning. Application programmers, library writers and compilers collaborate to describe and export a set of performance related tunable parameters to the Active Harmony system. These parameters define a tun- Active Harmony supports runtime adaptive code-generation and tuning for parameters that require new code (e.g. unroll factors). Effectively, we merge traditional feedback directed optimization and just-in-time compilation. This feature also enables application developers to write applications once and have the autotuner adjust the application behavior automatically when run on new systems. We evaluated our system on multiple large-scale parallel applications and showed that our system can improve the execution time by up to 46% compared to the original version of the program.Finally, we believe that the success of any auto-tuning research depends on how effectively application developers, domain-experts and auto-tuners communicate and work together. To that end, we have developed and released a simple and extensible language that standardizes the parameter space representation. Using this language, developers and researchers can collaborate to export tunable parameters to the tuning frameworks. Relationships (e.g. ordering, dependencies, constraints, ranking) between tunable parameters and search-hints can also be expressed.

show abstract

Parallel Parameter Tuning for Applications with Performance Variability

Cited by 30 publications

References 13 publications

Analytical Bounds for Optimal Tile Size Selection

Analytical Bounds for Optimal Tile Size Selection

Online Adaptive Code Generation and Tuning

Tuning parallel applications in parallel

Contact Info

Product

Resources

About