CROSS: A Multicriteria Group-Decision-Making Model for Evaluating and Prioritizing Advanced-Technology Projects at NASA

Abstract

Evaluating and prioritizing advanced-technology projects is a particularly difficult task for the staff at the Kennedy Space Center (KSC) shuttle project engineering office. Because the evaluation process is complex and unstructured, decision makers (DMs) must consider vast amounts of diverse information concerning safety, systems engineering, cost savings, process enhancement, reliability, and implementation. Intuitive methods developed in the past have helped them to use large volumes of information in evaluating projects. However, these intuitive methods do not provide a structured framework for systematic evaluation. CROSS (consensus-ranking organizational-support system) is a multicriteria group-decision-making model that I implemented successfully at KSC to capture the DMs' beliefs through sequential, rational, and analytical processes. CROSS uses the analytic hierarchy process (AHP), subjective probabilities, the entropy concept, and the maximize-agreement heuristic (MAH) to enhance the DMs' intuition in evaluating sets of projects. (Government: programs. Decision analysis: multiple criteria.) T he rapid development of technology over the last few decades and the increased awareness of its effects on society have focused critical attention on government agencies that support technology development. The public is concerned with the governance of these agencies and with obtaining the maximum return from public investment in advanced technology. Public pressure has forced Congress to mandate the National Aeronautic and Space Administration (NASA) to be more accountable in its evaluation of advanced-technology projects. The demand for accountability, the pressure to cut costs, and the increasing number of projects have made evaluating projects extremely difficult.Over the last several decades, analysts have developed a philosophy and a body of intuitive and analytical models to help decision makers (DMs) to evaluate and select projects. However, the intuitive models do not offer a structured framework for evaluating projects systematically, while the analytical models are not intended to capture intuitive preferences. The literature on project selection contains hundreds of models, including scoring methods, economic methods, portfolio methods, and decision analysis.Scoring methods use algebraic formulas to produce a score for each project (Lockett et al. 1984, Melachrinoudis and Rice 1991, Moore and Baker 1969. Economic methods use financial models to calculate the monetary payoff of each project under consideration (Graves and Ringuest 1991, Mehrez 1988). Portfolio methods evaluate the entire set

Project Portfolio Selection: A Newsvendor Approach

“…For example, the popular critical chain project management theory uses prioritizing projects to avoid the negative effects of multi‐tasking (Millhiser & Szmerekovsky, ) and not correctly prioritizing improvement projects is seen as a reason why improvement programs fail (Chakravorty, ). Further, much effort is put into prioritizing projects across a variety of settings, including manufacturing in distributed environments (Henriksen, ), advanced technology projects at NASA (Tavana, ), and government utilities capital improvement projects (Nagal & Elenbaas, ). Given the popularity of advocating prioritization of projects it seems highly promising to suspect that higher priority projects should get better funding, i.e., a lower probability of running over budget.…”

Are Formal Technology Integration Processes Needed for Successful Product Innovations?

“…The strategic practice of technology evaluation within a firm is cited as critical to innovation (Pavitt, 1984(Pavitt, , 2002Tidd and Bodley, 2002;Bond, 2003;Olshavsky and Spreng, 1996;Bessant et al, 2005;Brown and Utterback, 1985;Johne and Snelson, 1988). Several methodologies ranging from simple to advanced are available (Sarkis and Talluri, 1999;Delbecq et al, 1975;Scott, 2000;Torkkeli and Tuominen, 1997;Green and Gavin, 1995;Linstone and Turoff, 1975;Saaty and Vargas, 1991;Pistorius and Utterback, 1997;Roberts and Bellotti, 2002;Astebro, 2004;Christensen et al, 1998;Blackman Jr., 1986;Green et al, 1995;Heidenberger and Stummer, 1999;Tavana, 2003;Groenveld, 1997;Jagle, 1999;Bowonder, 1998;Mikkola, 2001;Stummer and Heidenberger, 2003;Souitaris, 2002;Kleindorfer and Partovi, 1990;Benson et al, 1993;Stummer et al, 2003;Cooper et al, 2000;Vepsalainen and Lauro, 1988;Kash and Rycroft, 2004). Schot et al (1997), Van-Eijndhoven (1997), Coates (1998), Van-Den-Ende et al (1998), Berloznik et al (1998…”

Linking Technology and New Product Development

“…For instance, quantitative methods are applied mostly for selecting or prioritizing research and development (R&D) projects or technologies [47,48] and developing product-technology roadmaps [49,50]. Qualitative methods for technology evaluation are mostly used for managing a strategic R&D portfolio [51][52][53].…”