Transferring Performance Prediction Models Across Different Hardware Platforms

Valov, Pavel; Petkovich, Jean-Christophe; Guo, Jianmei; Fischmeister, Sebastian; Czarnecki, Krzysztof

doi:10.1145/3030207.3030216

Cited by 51 publications

(81 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the use of transfer learning (cf. Figure 1) has been suggested to decrease the cost of learning by transferring knowledge about performance behavior across environments [7], [25], [51]. Similar to humans that learn from previous experience and transfer the learning to accomplish new tasks easier, here, knowledge about performance behavior gained in one environment can be reused effectively to learn models for changed environments with a lower cost.…”

Section: Introductionmentioning

confidence: 99%

Transfer learning for performance modeling of configurable systems: An exploratory analysis

Jamshidi

Siegmund

Velez

et al. 2017

2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

Abstract-Modern software systems provide many configuration options which significantly influence their non-functional properties. To understand and predict the effect of configuration options, several sampling and learning strategies have been proposed, albeit often with significant cost to cover the highly dimensional configuration space. Recently, transfer learning has been applied to reduce the effort of constructing performance models by transferring knowledge about performance behavior across environments. While this line of research is promising to learn more accurate models at a lower cost, it is unclear why and when transfer learning works for performance modeling. To shed light on when it is beneficial to apply transfer learning, we conducted an empirical study on four popular software systems, varying software configurations and environmental conditions, such as hardware, workload, and software versions, to identify the key knowledge pieces that can be exploited for transfer learning. Our results show that in small environmental changes (e.g., homogeneous workload change), by applying a linear transformation to the performance model, we can understand the performance behavior of the target environment, while for severe environmental changes (e.g., drastic workload change) we can transfer only knowledge that makes sampling more efficient, e.g., by reducing the dimensionality of the configuration space.

show abstract

Section: Introductionmentioning

confidence: 99%

Transfer learning for performance modeling of configurable systems: An exploratory analysis

Jamshidi

Siegmund

Velez

et al. 2017

2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

show abstract

“…[25] and Valov et al. [24]. Approach: We perform transfer learning the methods proposed by Valov et al.…”

Section: Rq4: How Does Beetle Compare To State-of-the-art Methods?mentioning

confidence: 99%

“…Approach: We perform transfer learning the methods proposed by Valov et al. [24] and Jamshidi et al. [25] (see §5).…”

Section: Rq4: How Does Beetle Compare To State-of-the-art Methods?mentioning

confidence: 99%

“…Valov et al. [24] proposed an approach for transferring performance models of software systems across platforms with different hardware settings. The method consists of the following two components: was found to provide good approximations of the transfer function.…”

Section: Transfer Learning With Linear Regressionmentioning

confidence: 99%

“…However, obtaining these performance measurements can cost a significant amount of time and money. For example, in one of the software systems studied here (X264), it takes over 1536 hours to obtain performance measurements for 11 out the 16 possible configuration options [24]. This is in addition to other time-consuming tasks involved in commissioning these systems such as setup, tear down, etc.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Whence to Learn? Transferring Knowledge in Configurable Systems Using BEETLE

Krishna

Nair

Jamshidi

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

As software systems grow in complexity and the space of possible configurations increases exponentially, finding the near-optimal configuration of a software system becomes challenging. Recent approaches address this challenge by learning performance models based on a sample set of configurations. However, collecting enough sample configurations can be very expensive since each such sample requires configuring, compiling, and executing the entire system using a complex test suite. When learning on new data is too expensive, it is possible to use Transfer Learning to "transfer" old lessons to the new context. Traditional transfer learning has a number of challenges, specifically, (a) learning from excessive data takes excessive time, and (b) the performance of the models built via transfer can deteriorate as a result of learning from a poor source. To resolve these problems, we propose a novel transfer learning framework called BEETLE, which is a "bellwether"-based transfer learner that focuses on identifying and learning from the most relevant source from amongst the old data. This paper evaluates BEETLE with 57 different software configuration problems based on five software systems (a video encoder, an SAT solver, a SQL database, a high-performance C-compiler, and a streaming data analytics tool). In each of these cases, BEETLE found configurations that are as good as or better than those found by other state-of-the-art transfer learners while requiring only a fraction ( 1 7 th) of the measurements needed by those other methods. Based on these results, we say that BEETLE is a new high-water mark in optimally configuring software.

show abstract

Regression Models for Performance Ranking of Configurable Systems: A Comparative Study

Chen

et al. 2020

Structured Object-Oriented Formal Language and Method

View full text Add to dashboard Cite

Transferring Performance Prediction Models Across Different Hardware Platforms

Cited by 51 publications

References 11 publications

Transfer learning for performance modeling of configurable systems: An exploratory analysis

Transfer learning for performance modeling of configurable systems: An exploratory analysis

Whence to Learn? Transferring Knowledge in Configurable Systems Using BEETLE

Regression Models for Performance Ranking of Configurable Systems: A Comparative Study

Contact Info

Product

Resources

About