Off-Road Performance Modeling – How to Deal with Segmented Data

Ilyas, Muhammad; Calotoiu, Alexandru; Wolf, Felix

doi:10.1007/978-3-319-64203-1_3

Cited by 9 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second step, Extra-P's model generator determines the best models that describe the training data points by regression, cross-validation, and iterative refinement (cf. Calotoiu et al 20,21 for details). The regression tries to minimize the residual sum of squares (RSS) between the training data points and the model function which is defined as:…”

Section: Model Generationmentioning

confidence: 99%

“…where p is the input parameter whose influence is considered and c 0 a constant term. Reisert et al 21 have shown that in most cases 𝜈 = 1 is sufficient to achieve a good result. That means the model looks like:…”

Section: Model Generationmentioning

confidence: 99%

“…For our own metric anRSS, we start from the absolute normalized RSS (nRSS). This was presented by Ilyas et al 21 and is defined as:…”

Section: Quality Assessmentmentioning

confidence: 99%

See 2 more Smart Citations

SimAnMo—A parallelized runtime model generator

Burger

Nguyen

Bischof

2021

Concurrency and Computation

View full text Add to dashboard Cite

In this article, we present the novel features of the recent version of SimAnMo, the Simulated Annealing Modeler. The tool creates models that correlate the size of one input parameter of an application to the corresponding runtime and thus SimAnMo allows predictions for larger input sizes. A focus lies on applications whose runtime grows exponentially in the input parameter size. Such programs are, for example, of high interest for cryptanalysis to analyze practical security of traditional and post-quantum secure schemes. However, SimAnMo also generates reliable models for the widespread case of polynomial runtime behavior and also for the important case of factorial runtime increase. SimAnMo's model generation is based on a parallelized simulated annealing procedure and heuristically minimizes the costs of a model. Those may rely on different quality metrics. Insights into SimAnMo's software design and its usage are provided. We demonstrate the quality of SimAnMo's models for different algorithms from various application fields. We show that our approach also works well on ARM architectures.

show abstract

Section: Model Generationmentioning

confidence: 99%

Section: Model Generationmentioning

confidence: 99%

See 1 more Smart Citation

SimAnMo—A parallelized runtime model generator

Burger

Nguyen

Bischof

2021

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…We notice a qualitative, not merely a quantitative difference between execution on 4, 8, 16 and larger numbers of ranks. As there is more than one behavior to be modeled in one interval, the parametric models estimated by Extra-P cannot represent the function accurately unless more measurement data is provided [25].…”

Section: Validitymentioning

confidence: 99%

Extracting Clean Performance Models from Tainted Programs

Copik¹,

Calotoiu²,

Grosser³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Performance models are well-known instruments to understand the scaling behavior of parallel applications. They express how performance changes as key execution parameters, such as the number of processes or the size of the input problem, vary. Besides reasoning about program behavior, such models can also be automatically derived from performance data. This is called empirical performance modeling. While this sounds simple at the first glance, this approach faces several serious interrelated challenges, including expensive performance measurements, inaccuracies inflicted by noisy benchmark data, and overall complex experiment design, starting with the selection of the right parameters. The more parameters one considers, the more experiments are needed and the stronger the impact of noise. In this paper, we show how taint analysis, a technique borrowed from the domain of computer security, can substantially improve the modeling process, lowering its cost, improving model quality, and help validate performance models and experimental setups.

show abstract

“…For example, linear functions can be used to describe the influence of the numeric configuration options of TRIMESH over their whole value domain [62]. In contrast, in some systems, there are options with different influences in disjoint parts of the value domain, as described by Ilyas et al [63] or by Courtois and Woodside [64]. For these options, different functions have to be learned for different parts of their value domain.…”

Section: B Comparison Of Sampling Strategiesmentioning

confidence: 99%

Predicting Performance of Software Configurations: There is no Silver Bullet

Grebhahn¹,

Siegmund²,

Apel³

2019

Preprint

View full text Add to dashboard Cite

Many software systems offer configuration options to tailor their functionality and non-functional properties (e.g., performance). Often, users are interested in the (performance-)optimal configuration, but struggle to find it, due to missing information on influences of individual configuration options and their interactions. In the past, various supervised machine-learning techniques have been used to predict the performance of all configurations and to identify the optimal one.In the literature, there is a large number of machine-learning techniques and sampling strategies to select from. It is unclear, though, to what extent they affect prediction accuracy. We have conducted a comparative study regarding the mean prediction accuracy when predicting the performance of all configurations considering 6 machine-learning techniques, 18 sampling strategies, and 6 subject software systems.We found that both the learning technique and the sampling strategy have a strong influence on prediction accuracy. We further observed that some learning techniques (e.g., random forests) outperform other learning techniques (e.g., k-nearest neighbor) in most cases. Moreover, as the prediction accuracy strongly depends on the subject system, there is no combination of a learning technique and sampling strategy that is optimal in all cases, considering the tradeoff between accuracy and measurement overhead, which is in line with the famous no-free-lunch theorem.

show abstract

Off-Road Performance Modeling – How to Deal with Segmented Data

Cited by 9 publications

References 9 publications

SimAnMo—A parallelized runtime model generator

SimAnMo—A parallelized runtime model generator

Extracting Clean Performance Models from Tainted Programs

Predicting Performance of Software Configurations: There is no Silver Bullet

Contact Info

Product

Resources

About