The Development of Theory-Driven Evaluation in the Military

Abstract

The use of theory-driven evaluation is an emerging practice in the military-an aspect generally unknown in the civilian evaluation community. First developed during the 1991 Gulf War and applied in both the Balkans and Afghanistan, these techniques are now being examined in the North Atlantic Treaty Organisation (NATO) as a means to evaluate the effects of military operations in complex, asymmetric conflict environments. In spite of these practices, theory-driven evaluation in the military is still in the developmental stages. This article traces the development to date of theory-driven evaluation in NATO and assesses its strengths and weaknesses in the military context. We conclude that a cross-pollination of ideas between military and civilian evaluators is urgently needed to improve the quality and effectiveness of military evaluation. F rom the development of methodologies to test the effectiveness of various educational initiatives in schools and universities in the early 20th century, the field of evaluation has evolved and grown into a discipline of study in its own right, and one that interlinks with many research disciplines including: Social science, public administration, public policy, and education. Throughout the world, it is now commonplace to find whole evaluation departments integrated with government agencies, nonprofit, nongovernmental, and international organizations. Furthermore, within each respective organization, the role of evaluation typically has a place throughout the entire policy and service delivery process, from the identification of a public need to ensuring the value and quality of final, implemented solutions (Rogers, 2007).The processes involved in planning, executing, and evaluating a military operation bear many similarities to that of planning, implementing and evaluating projects and services in civilian organizations. At the simplest level, in both civilian and military domains, a problem is identified that requires some action to be taken to alter the situation and rectify the problem. Although the idea of assessing both the progress and the impact of this action has been relatively well understood and extensively studied in civilian settings, the concept is surprisingly underdeveloped in the military domain.

Evaluating the Implementation of a Professional Sport Team’s Corporate Community Involvement Initiative

“…Determining quality standards and priorities for improvement helped in the evaluation because it allowed for a focused perspective on where and how to improve program implementation. Second, program theory evaluation (Chen, 2005) seeks to first examine how a program is delivered and then determine the congruency between planned program implementation with the actual implemented program (Chen, 2005;Williams & Morris, 2009). Stakeholder views are sought to help evaluators ascertain within the particular program context how the program should be delivered and where expectations were not met.…”

A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009

“…In the past few years, theory-driven approaches also have been increasingly promoted in international development settings, including some of the evaluations commissioned and conducted by the Overseas Development Institute (ODI), the International Initiative for Impact Evaluation (3ie), the United Nations Evaluation Group (UNEG), and the Independent Evaluation Group (IEG) of the World Bank for evaluating humanitarian efforts (Carvalho & White, 2004;Conlin & Stirrat, 2008;White, 2007White, , 2009; White & Masset, 2007;World Bank, 2003. More recently, such approaches have been suggested as a means for evaluating military operations in the United States (Williams & Morris, 2009) as well as in a variety of other fields, settings, and contexts (Trochim, Marcus, Masse, Moser, & Weld, 2008;Urban & Trochim, 2009;Weiss, 1997b). Theorydriven forms of evaluation also have been recommended as one possible alternative to randomized controlled trials or randomized experiments-generally patterned after the evidence-based practice model in medicine-for both independent and federally sponsored initiatives charged with identifying efficacious or effective interventions (Government Accountability Office [GAO], 2009).…”

Evaluation theory and its application to military assessments

“…In official guidance, Department of Defense instructions explicitly recommend the use of evaluation theoretic concepts in defense assessments. 10 Conversely, the evaluation community has also recognized the link between military assessment and evaluation, as in Williams and Morris 11 .…”