Customer Involvement in Continuous Deployment: A Systematic Literature Review

Yaman, Sezin Gizem; Sauvola, Tanja; Riungu-Kalliosaari, Leah; Hokkanen, Laura; Kuvaja, Pasi; Oivo, Markku; Männistö, Tomi

doi:10.1007/978-3-319-30282-9_18

Cited by 18 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experiments can be executed either on unreleased prototypes or after deployment to real users. Bosch-Sijtsema and Bosch (2015) and Yaman et al (2016) explain how qualitative experiments on prototypes are used early in development to validate overarching assumptions about user experience and the business model. While post-deployment experiments, such as randomized controlled experiments, are used to measure small differences between software variants for optimizing a business related or user experience (UX) metric.…”

Section: Background and Related Work On Continuous Experimentationmentioning

confidence: 99%

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

Ros

Hammar²

2020

Empir Software Eng

View full text Add to dashboard Cite

Context Software design in e-commerce can be improved with user data through controlled experiments (i.e. A/B tests) to better meet user needs. Machine learning-based algorithmic optimization techniques extends the approach to large number of variables to personalize software to different user needs. So far the optimization techniques has only been applied to optimize software of low complexity, such as colors and wordings of text. Objective In this paper, we introduce the COMBO toolkit with capability to model optimization variables and their relationship constraints specified through an embedded domain-specific language. The toolkit generates personalized software configurations for users as they arrive in the system, and the configurations improve over time in in relation to some given metric. COMBO has several implementations of machine learning algorithms and constraint solvers to optimize the model with user data by software developers without deep optimization knowledge. Method The toolkit was validated in a proof-of-concept by implementing two features that are relevant to Apptus, an e-commerce company that develops algorithms for web shops. The algorithmic performance was evaluated in simulations with realistic historic user data. Results The validation shows that the toolkit approach can model and improve relatively complex features with many types of variables and constraints, without causing noticeable delays for users. Conclusions We show that modeling software hierarchies in a formal model facilitates algorithmic optimization of more complex software. In this way, using COMBO, developers can make data-driven and personalized software products.

show abstract

Section: Background and Related Work On Continuous Experimentationmentioning

confidence: 99%

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

Ros

Hammar²

2020

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…The growing size of data is a challenge, and systematic approaches for collecting, analyzing, and integrating data into the product development process are missing [14]. Moreover, there is a lack of methods and tools for integrating, analyzing, and visualizing collected data to make QRs transparent and support real-time decision-making on QRs [15], and to jointly manage QRs and FRs along the RSD process [4].…”

Section: Challenges In Managing Quality Requirementsmentioning

confidence: 99%

How Can Quality Awareness Support Rapid Software Development? – A Research Preview

Guzmán

Oriol

Rodríguez

et al. 2017

Requirements Engineering: Foundation for Software Quality

Self Cite

View full text Add to dashboard Cite

Context: Rapid software development (RSD) refers to the organizational capability to develop, release, and learn from software in rapid cycles without compromising its quality. To achieve RSD, it is essential to understand and manage software quality along the software lifecycle. Problem: Despite the numerous information sources related to product quality, there is a lack of mechanisms for supporting continuous quality management throughout the whole RSD process. Principal ideas/results: We propose Q-Rapids, a data-driven, qualityaware RSD methodology in which quality and functional requirements are managed together. Quality Requirements are incrementally elicited and refined based on data gathered at both development time and runtime. Project, development, and runtime data is aggregated into quality-related key indicators to support decision makers in steering future development cycles. Contributions: Q-Rapids aims to increase software quality through continuous data gathering and analysis, as well as continuous management of quality requirements.

show abstract

“…Based on our empirical findings [9], [36], [156] and learnings from other researchers [10], [104], [113], [179], both types of feedback have their profound impact in the software development process and are important for successful software product development. However, and although both types of feedback complement each other, there is a fundamental difference in their accuracy, and the types of conclusions that we can make when using either of them.…”

Section: Background 821 Learning What Customers Valuementioning

confidence: 60%

“…Companies can learn from the product usage data that they collect in several different ways [113]. One possibility is through exploratory data analysis, which helps understand the data when little or no hypotheses exist, or when specific hypotheses exist but supplemental representations are needed to ensure the interpretability of the results [114].…”

Section: Benefiting From Feedback Data Through CI and Cdmentioning

confidence: 99%

Data-Driven Software Development at Large Scale : from Ad-Hoc Data Collection to Trustworthy Experimentation

Fabijan¹

View full text Add to dashboard Cite

Accurately learning what customers value is critical for the success of every company. Despite the extensive research on identifying customer preferences, only a handful of software companies succeed in becoming truly data-driven at scale. Benefiting from novel approaches such as experimentation in addition to the traditional feedback collection is challenging, yet tremendously impactful when performed correctly. In this thesis, we explore how software companies evolve from data-collectors with ad-hoc benefits, to trustworthy data-driven decision makers at scale. We base our work on a 3.5-year longitudinal multiple-case study research with companies working in both embedded systems domain (e.g. engineering connected vehicles, surveillance systems, etc.) as well as in the online domain (e.g. developing search engines, mobile applications, etc.). The contribution of this thesis is threefold. First, we present how software companies use data to learn from customers. Second, we show how to adopt and evolve controlled experimentation to become more accurate in learning what customers value. Finally, we provide detailed guidelines that can be used by companies to improve their experimentation capabilities. With our work, we aim to empower software companies to become truly data-driven at scale through trustworthy experimentation. Ultimately this should lead to better software products and services.

show abstract

Customer Involvement in Continuous Deployment: A Systematic Literature Review

Cited by 18 publications

References 15 publications

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

How Can Quality Awareness Support Rapid Software Development? – A Research Preview

Data-Driven Software Development at Large Scale : from Ad-Hoc Data Collection to Trustworthy Experimentation

Contact Info

Product

Resources

About