A decision support framework for metrics selection in goal-based measurement programs: GQM-DSFMS

Abstract

Software organisations face challenges in managing and sustaining their measurement programs over time. The complexity of measurement programs increase with exploding number of goals and metrics to collect. At the same time, organizations usually have limited budget and resources for metrics collection. It has been recognized for quite a while that there is the need for prioritizing goals, which then ought to drive the selection of metrics. On the other hand, the dynamic nature of organisations require measurement programs to adapt to changes in the stakeholders, their goals, information needs and priorities. Therefore, it is crucial for organisations to use structured approaches that provide transparency, traceability and guidance in choosing an optimum set of metrics that would address the highest priority information needs considering limited resources. This paper proposes a decision support framework for metrics selection (DSFMS) which is built upon the widely-used Goal Question Metric (GQM) approach. The core of the framework includes an iterative goal-based metrics selection process incorporating decision making mechanisms in metrics selection, a pre-defined attributes/metrics repository, and a traceability model among GQM elements. We also discuss alternative prioritization and optimization techniques for organizations to tailor the framework according to their needs. The evaluation of the GQM-DSFMS framework was done through a case study in a CMMI Level 3 software company.

Success Factors for Effective Process Metrics Operationalization in Agile Software Development: A Multiple Case Study

“…Hall and Fenton [3] limit their discussion about data till automatic data collection, which we emphasize under the F1 sub-factor of Data format. We drew a similar conclusion from the study by Gencel et al [21], where the emphasis is to prioritize metrics and collect only relevant data, in response to an organization's limited resource. This is similar to Int2's comment about choosing only the 'must-have' metrics, and Int1's rationale for using a smaller set of metrics for operationalization.…”

“…The factor Metrics trustworthiness (F5) appears to be the most important factor, especially from the standpoint of longterm metrics programs use. The sub-factor of Familiarity has been raised by Gencel et al [21], but only from the utilitarian perspective of relying on existing standards for defining, using, and reusing metrics. However, our finding about how familiar metrics can help to engage target users and establish initial trust, which later translates into sustaining that trust, is underrepresented in existing literature.…”

“…Similarly, the automatic data collection plugin developed in Q-Rapids [12], [13] satisfies the factor HF9, aimed at minimizing efforts for data collection. Gencel et al [21] also emphasized the importance of data collection, and how an organization's resources can be a limiting factor.…”

Quality metrics in Agile Software Development Projects

“…During time a number of frameworks, models and standards have been developed to support software projects in planning their measurement approach [15]. Most of the approaches tries to provide cost-effective measurement methods by collecting only useful data through metrics with a clear purpose.…”

“…Most of the approaches tries to provide cost-effective measurement methods by collecting only useful data through metrics with a clear purpose. A large number of empirical studies have shown that goal-based measurement frameworks are effective and adaptable [15].…”