Cultivation and engineering of a software metrics program

Abstract

This paper reports from a case study of an organization that implements a software metrics program to measure the effects of its improvement efforts. The program measures key indicators of all completed projects and summarizes progress information in a quarterly management report. The implementation turns out to be long and complex, as the organization is confronted with dilemmas based on contradictory demands and value conflicts. The process is interpreted as a combination of a rational engineering process in which a metrics program is constructed and put into use, and an evolutionary cultivation process in which basic values of the software organization are confronted and transformed. The analysis exemplifies the difficulties and challenges that software organizations face when bringing known principles for software metrics programs into practical use. The article discusses the insights gained from the case in six lessons that may be used by Software Process Improvement managers in implementing a successful metrics program.

The Role of Conventional Research Methods in Information Systems Action Research

“…From Latour's perspective, Aaen et al addressed software process improvement as debatable, a relevant issue, and a contribution to an existing domain of knowledge. Based on the relevancy of the Aaen et al study, subsequent projects ensued, relying on additional methods in an emergent approach to determine the authenticity of software process improvement facts in the particular context of a metrics program and to find out how these facts should be pursued (Iversen and Mathiassen 2003;Iversen et al 1999). We believe that other lenses could also be used to illustrate how the planned and emergent approaches are used together including Lee's (1991) subjective and objective research approaches and Orlikowski and Baroudi's (1991) description of varied ontologies.…”

“…Two examples illustrate this. Iversen and Mathiassen (2003) present a traditional case study based on interviews, documents, minutes of meetings, etc., in an attempt to design and implement a software metrics program in one of the four organizations. Action research played no role in the development of this particular publication, but the action research approach to the overall project created the opportunity for the involved researchers to identify and develop this research.…”

Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review

“…First, the focus of this review is on measurement of software process improvement initiatives, i. e. what to measure, and is therefore more specific than the reviews of Bellini et al and Gomez et al Second, this review investigates also how the measures are used to evaluate and analyze the process improvement. Given our different focus, only 1 ( [185]) of our 148 reviewed papers was also covered by Bellini et al [10]. Gomez et al [48] did not report the reviewed papers which impedes a coverage assessment with our SLR.…”

Assessing Improvements of Software Metrics Practices

“…Section 2.1 gives an overview of what software metrics are and recommendations on how to implement them. Failing to effectively support software management a metrics program will easily degenerate into a bureaucratic procedure, where data are collected, stored, and reported without having practical consequences (Iversen & Kautz, 2001;Iversen & Mathiassen, 2003;Niessink & Vliet, 1999). Hence, we look in Section 2.2 at different ways in which metrics programs can serve as control mechanisms and in Section 2.3 at the system of organizational mechanisms that are needed to support their effective operation.…”

“…It has been estimated that as many as 78% of metrics programs fail (Dekkers, 1999) and recent research has consequently addressed success factors, e.g. (Hall & Fenton, 1997;Iversen & Mathiassen, 2003). Niessink and Vliet (1999) stress focusing on the actual use of a metrics program to make sure that it becomes of value to the organization.…”