Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Abstract

With large scale and complex configurable systems, it is hard for users to choose the right combination of options (i.e., configurations) in order to obtain the wanted trade-off between functionality and performance goals such as speed or size. Machine learning can help in relating these goals to the configurable system options, and thus, predict the effect of options on the outcome, typically after a costly training step. However, many configurable systems evolve at such a rapid pace that it is impractical to retrain a new model from scratch for each new version. In this paper, we propose a new method to enable transfer learning of binary size predictions among versions of the same configurable system. Taking the extreme case of the Linux kernel with its ≈ 14, 500 configuration options, we first investigate how binary size predictions of kernel size degrade over successive versions. We show that the direct reuse of an accurate prediction model from 2017 quickly becomes inaccurate when Linux evolves, up to a 32% mean error by August 2020. We thus propose a new approach for transfer evolution-aware model shifting (TEAMS). It leverages the structure of a configurable system to transfer an initial predictive model towards its future versions with a minimal amount of extra processing for each version. We show that TEAMS vastly outperforms state of the art approaches over the 3 years history of Linux kernels, from 4.13 to 5.8.

Transferring Performance between Distinct Configurable Systems : A Case Study

“…As many other transfer learning works [4,24], it is applied to transfer performance of executing environments. Martin et al develop TEAMs [21], a transfer learning approach predicting the performance distribution of the Linux kernel using the measurements of its previous releases. Between two releases, related the same system but distant in time, one could consider that it is a simple case of transfer across systems.…”

“…Software offers more and more options that users can (de)select to customize the system for their specific needs. With the exploding number of options e.g., +2041 options in three years for the Linux kernel [21], it becomes complex to accurately estimate the individual impact of options, difficult to predict software performance and unthinkable to measure exhaustively the configuration space of real-world systems. To overcome this problem, related work has proposed to train machine learning models a.k.a.…”

“…But this method has a cost [25]: since we feed the model with configurations and performance, we have to measure numerous configurations. Transfer learning techniques [21,24] aim at reducing this measurement cost: if a performance model has already been trained on one executing environment, we can reuse this model to reduce the number of measurements needed for a second executing environment [13], assuming that both environments are similar (e.g., same operating system). We propose to apply transfer learning to distinct software systems performing the same task, such as compilers (e.g., gcc and llvm), container managers (e.g., podman and docker), etc.…”

Feature subset selection for learning huge configuration spaces

“…There have been recent attempts about learning the configuration space of the Linux Kernel [42,43]. We consider the problem of predicting properties of any configuration, whereas Martin et al [43] consider the problem of specializing the configuration space.…”

“…Another key difference of our work is that we study in detail the quality of feature selection: (1) how many features are needed to reach a good tradeoff between accuracy, interpretability, and computation time; (2) how features considered as influential relate to domain knowledge. In [42], Martin et al learn performance models over the binary size of different Linux versions (from 4.13 to 5.8). In contrast to our study, feature selection and learning over a reduced set of options are not considered.…”