A Framework for Studying Cost Growth on Complex Acquisition Programs

Abstract

The government acquisition system is consistently plagued by cost growth and by attempts at acquisition reform. Despite these persistent challenges, the academic community lacks a methodology for studying complex acquisition programs both in‐depth and longitudinally throughout their life cycles. In this paper, we present a framework for studying complex acquisition programs that provides researchers with a strategy for systematically studying cost growth mechanisms. The proposed framework provides a means to identify specific technical and organizational mechanisms for cost growth, to organize those mechanisms using design structure matrices, and to observe the evolution of those mechanisms throughout a program's life cycle. To illustrate the utility of our framework, we apply it to analyze a case study of the National Polar‐orbiting Operational Environmental Satellite System (NPOESS) program. Ultimately, we demonstrate that the framework enables us to generate unique insights into the mechanisms that induced cost growth on NPOESS and that were unacknowledged by previous studies. Specifically, we observed that complexity was injected into the technical system well before the program's cost estimates began to increase and that it was the complexity of the NPOESS organization that hindered the program's ability to effectively estimate and to manage its costs.

Qualitative methods for engineering systems: Why we need them and how to use them

“…Only recently have studies that employ qualitative methods begun to be published in the journal Systems Engineering . For example, Dwyer et al . uses a mixed methods approach to understand how organizational and technical systems interact to generate cost growth in the acquisition of complex systems.…”

“…Neither author sees herself as a methodologist. We are writing this paper as a translation of established literature from other domains, filtered through the lens of our hard‐learned lessons from applying these methods to systems engineering questions . Our hope is that by framing the key issues as clearly as we can, other engineers who hope to join the qualitative path will build on what we have learned so far and bring increased rigor to their pursuits.…”

Towards a solver-aware systems architecting framework: leveraging experts, specialists and the crowd to design innovative complex systems

“…This incentivizes practicing system architects to adopt an uncertainty reducing approach, where the abstract concept is iteratively refined by articulating its inputs, outputs, and processes, along with the interfaces through which these interactions will occur (Kossiakoff & Sweet 2003; Larson et al 2009; Buede & Miller 2016). The concept development is followed by an exploration of the design space (Hazelrigg 1998; Du & Chen 2002; Ross et al . 2008; Collopy & Hollingsworth 2011; Topcu & Mesmer 2018).…”

Evaluating the complexity of engineered systems: A framework informed by a user case study

“…As a consequence, the development of unambiguous and reliable measures of system complexity is a considerable challenge. Metrics such as cyclomatic complexity 60 and lines of code 61 have been used in software engineering to measure software complexity, and the number and connectivity of physical system components and interfaces are used to measure the complexity of a product architecture topology 32,36,62–65 . However, developing metrics for a diverse system as a whole remains a challenge 66 .…”