An empirical study on principles and practices of continuous delivery and deployment

Abstract

Despite substantial recent research activity related to continuous delivery and deployment (CD), there has not yet been a systematic, empirical study on how the practices often associated with continuous deployment have found their way into the broader software industry. This raises the question to what extent our knowledge of the area is dominated by the peculiarities of a small number of industrial leaders, such as Facebook. To address this issue, we conducted a mixed-method empirical study, consisting of a pre-study on literature, qualitative interviews with 20 software developers or release engineers with heterogeneous backgrounds, and a Web-based quantitative survey that attracted 187 complete responses. A major trend in the results of our study is that architectural issues are currently one of the main barriers for CD adoption. Further, feature toggles as an implementation technique for partial rollouts lead to unwanted complexity, and require research on better abstractions and modelling techniques for runtime variability. Finally, we conclude that practitioners are in need for more principled approaches to release decision making, e.g., which features to conduct A/B tests on, or which metrics to evaluate.

An empirical analysis of the Docker container ecosystem on GitHub

“…Such long build durations are especially problematic if the step to build the image is part of a continuous deployment or delivery pipeline [11]. The idea of fast, frequent releases [12] with immediate customer feedback is impeded by long lasting build processes. Docker's strategy of splitting images into multiple layers addresses this problem and speeds up the process by only downloading those resources (e.g., packages) which are not yet available locally.…”

Towards quality gates in continuous delivery and deployment

“…There has been a multitude of research on the field of selective regression testing [8]. However, as we learned throughout the interviews of our two studies [2], [3], none of the companies used any tool resulting from this research. Instead, they either executed all tests or a subset based on intuition.…”

“…CD influences many phases of traditional development processes [5] and is associated with several risk mitigation strategies for keeping the impact of issues low [3].…”

“…The idea for this work evolved from two recently conducted studies -an analysis of the impact of cloud computing on software development processes [2], and a study on how the practices associated with continuous delivery and deployment found their way into the broad software industry [3]. In both studies, we conducted qualitative interviews with 25 and 20 professional software developers or release engineers, respectively, and validated the results by quantitative surveys with more than 450 (>290 and >180) practitioners in total.…”

Bifrost

“…In cloud-based software engineering, practices such as Dev-Ops, continuous delivery, and continuous deployment, have recently reached mainstream acceptance in the developer community. A common feature of these practices is that they provide means for software houses to further speed up their release processes and to get their products into the hands of their users faster [22]. For cloud-based Software-asa-Service (SaaS) applications, this idea of "releasing faster" often comes in the form of wide-ranging automation, e.g., a deployment pipeline [2] that, fully automatedly, builds, tests, and pushes changes into production.…”