Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software

Xu, Tianyin; Jin, Long; Fan, Xuepeng; Zhou, Yuanyuan; Pasupathy, Shankar; Talwadker, Rukma

doi:10.1145/2786805.2786852

Cited by 142 publications

(100 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With container image repositories, the usage statistics of configuration parameters can be collected by analyzing the configuration files in the image repositories built for the same piece of software. For popular software (e.g., those studied in [25]), there are typically thousands of image repositories made for different use cases and scenarios, containing a diverse set of configuration settings. 4 Moreover, as image repositories contain different versions of the target software and the configurations working for each version, mining these repositories enables the opportunities to understand software configuration with software evolution in depth.…”

Section: Opportunitiesmentioning

confidence: 99%

“…Historically, studying field configurations used ethnographic methods [3,8] and manual data collection from second-hand data sources [25,27]. For example, a study of how software is configured in the field [25] took six person-month to collect configuration files attached in issue reports on mailing lists and online forums. However, this dataset, despite the only one of its kind, is highly biased to misconfiguration cases and is incomplete-it is hard to determine the values referencing to execution context (e.g., environment variables and file content).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mining container image repositories for software configuration and beyond

Marinov

2018

Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results

Self Cite

View full text Add to dashboard Cite

This paper introduces the idea of mining container image repositories for configuration and other deployment information of software systems. Unlike traditional software repositories (e.g., source code repositories and app stores), image repositories encapsulate the entire execution ecosystem for running target software, including its configurations, dependent libraries and components, and OS-level utilities, which contributes to a wealth of data and information. We showcase the opportunities based on concrete software engineering tasks that can benefit from mining image repositories. To facilitate future mining efforts, we summarize the challenges of analyzing image repositories and the approaches that can address these challenges. We hope that this paper will stimulate exciting research agenda of mining this emerging type of software repositories. CCS CONCEPTS• Software and its engineering → Software libraries and repositories; Software post-development issues; Software configuration management and version control systems;

show abstract

Section: Opportunitiesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Mining container image repositories for software configuration and beyond

Marinov

2018

Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results

Self Cite

View full text Add to dashboard Cite

show abstract

“…Here, |N | is the number of configuration options available in the software system. If the options for each configuration is binary, then there can be as much as 2 |N | possible configurations for a given system 1 configurations, we measure the performance of a subset of the 2 |N | samples, this subset is denoted by |C|. The performance of each of the |C| configurations are measured under different hardware (H), workloads (W ), and software versions (V ).…”

Section: Subject Systemsmentioning

confidence: 99%

“…1: Traditional Transfer Learning compared with using bellwethers to discover near optimal configurations. by other state-of-the-art transfer learners while requiring only 1 7 -th of the measurements needed by those other methods. Reducing the number of measurements is an important consideration since collecting data in this domain can be computationally and monetarily expensive.…”

Section: Introductionmentioning

confidence: 99%

Whence to Learn? Transferring Knowledge in Configurable Systems Using BEETLE

Krishna

Nair

Jamshidi

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

As software systems grow in complexity and the space of possible configurations increases exponentially, finding the near-optimal configuration of a software system becomes challenging. Recent approaches address this challenge by learning performance models based on a sample set of configurations. However, collecting enough sample configurations can be very expensive since each such sample requires configuring, compiling, and executing the entire system using a complex test suite. When learning on new data is too expensive, it is possible to use Transfer Learning to "transfer" old lessons to the new context. Traditional transfer learning has a number of challenges, specifically, (a) learning from excessive data takes excessive time, and (b) the performance of the models built via transfer can deteriorate as a result of learning from a poor source. To resolve these problems, we propose a novel transfer learning framework called BEETLE, which is a "bellwether"-based transfer learner that focuses on identifying and learning from the most relevant source from amongst the old data. This paper evaluates BEETLE with 57 different software configuration problems based on five software systems (a video encoder, an SAT solver, a SQL database, a high-performance C-compiler, and a streaming data analytics tool). In each of these cases, BEETLE found configurations that are as good as or better than those found by other state-of-the-art transfer learners while requiring only a fraction ( 1 7 th) of the measurements needed by those other methods. Based on these results, we say that BEETLE is a new high-water mark in optimally configuring software.

show abstract

“…For example, the scitkit-learn toolkit lists over a dozen configuration options for Logistic Regression 4 . This is an important point since recent research shows that the more settings we add to software, the harder it becomes for humans to use that software [105]. -Manually fine-tuning these parameters to obtain best results for a new data set, is not only tedious, but also can be biased by a human's (mis-)understanding of the inner workings of the data miner.…”

Section: Claim2: Optimizers Can Greatly Improve Data Minersmentioning

confidence: 99%

Better software analytics via “DUO”: Data mining algorithms using/used-by optimizers

et al. 2020

View full text Add to dashboard Cite

This paper claims that a new field of empirical software engineering research and practice is emerging: data mining using/used-by optimizers for empirical studies, or DUO. For example, data miners can generate the models that are explored by optimizers. Also, optimizers can advise how to best adjust the control parameters of a data miner. This combined approach acts like an agent leaning over the shoulder of an analyst that advises "ask this question next" or "ignore that problem, it is not relevant to your goals". Further, those agents can help us build "better" predictive models, where "better" can be either greater predictive accuracy, or faster modeling time (which, in turn, enables the exploration of a wider range of options). We also caution that the era of papers that just use data miners is coming to an end. Results obtained from an unoptimized data miner can be quickly refuted, just by applying an optimizer to produce a different (and better performing) model. Our conclusion, hence, is that for software analytics it is possible, useful and necessary to combine data mining and optimization using DUO.

show abstract

Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software

Cited by 142 publications

References 50 publications

Mining container image repositories for software configuration and beyond

Mining container image repositories for software configuration and beyond

Whence to Learn? Transferring Knowledge in Configurable Systems Using BEETLE

Better software analytics via “DUO”: Data mining algorithms using/used-by optimizers

Contact Info

Product

Resources

About