Success Factors for Effective Process Metrics Operationalization in Agile Software Development: A Multiple Case Study

Abstract

Existing literature proposes success factors for establishing metrics programs. However, very few studies focus on factors that could ensure long-term use of metrics, and even fewer studies investigate such factors in the context of Agile Software Development (ASD). Motivated by this knowledge gap, we aim to identify success factors for operationalizing metrics in ASD, particularly, factors that could help in the long-term use of metrics. We conducted a multiple case study, where we operationalized process metrics at two software-intensive companies using ASD. We learned that data availability and development process are the two fundamental success factors for process metrics operationalization, albeit less prominent in literature. Companies prefer iterative and incremental operationalization of stable and functional process metrics, which is analogous to the agile way of working. Metrics trustworthiness plays a key role in successful operationalization of process metrics, and is potentially vital to ensuring their longterm use. By comparing the identified success factors with the existing literature, we conclude that success factors concerning data availability, development process, and metrics trustworthiness warrant greater attention, especially to maximize the chances of long-term use of process metrics.

Hybrid and evolving processes for software and systems—ICSSP 2019 special issue

“…We further learned in this session that ProDynamics feedback can lead to a performance increase through sprints, helping to reduce problems with workload estimation 10 . Finally, research on success factors of process metrics was presented, showing that data availability, development process, and metrics trustworthiness are crucial when it comes to long‐term use of process metrics 11 …”

Actionable Software Metrics

“…The case study in [20] highlight the need for a metric to be reliable to make them actionable. Similarly, the case study in [17] highlight five success factors for effective operationalization of an MP, and the authors emphasize on metrics trustworthiness in order for practitioners to find them useful. Therefore, beyond a metric's reliability, a product of metric's trustworthiness, the above-mentioned studies do not tackle what makes a metric actionable.…”

“…CC2 participants could act on the 'well-defined issues jira' metric, as they found them simple and even reliable. In addition, the straightforward calculation for the metric enabled them to become familiar with it more easily, which is one of the success factors for use and sustainability of an MP [17]. A simple metric and a reliable metric are not necessarily mutually inclusive, but the former heavily influences the latter.…”

“…Software metrics can be used to empower software practitioners to gain and share insights from their data for better decision-making [13,25]. In addition to being trustworthy [17] or reliable [20], such metrics need to manifest practitioners' practical requirements in order to contribute any actionable information [25]. Essentially, a metric that can influence action [2] to facilitate process improvement or remedy a situation [23] is an actionable metric.…”

“…However, literature suggests that a metric is actionable only in the latter case [2]. There are studies that recommend success factors for a metrics program [17,20,21], and suggest measures to increase responsiveness to customers' needs [15], while broaching the topic of using metrics for taking actions. However, these studies do not discuss what makes a metrics truly actionable, beyond their requirement to be reliable [20].…”