Evaluating the Quality of Structured Environmental Management Decisions

Abstract

Structured decision making (SDM) approaches have been advocated as a means of improving the quality of environmental and related risk management decisions based largely on the self-reported behavior of decision makers. The goal of the research presented here was to test this basis for decision quality by comparing the selfreported assessments of individual decision makers with their actual choice behavior across a set of three related environmental contexts. It was hypothesized that a modified structured decision approach would lead participants to make better informed decisions which accurately reflected their objectives, not based solely on self-reports, but also on internally consistent decision making behavior. Results from this study only partially support this hypothesis. While individuals' self-reports suggest that the structured approach outperformed results from an unstructured control condition, there was a lack of agreement between these self-reported evaluations and actual choice behavior. Beyond the obvious policy implications of decisions that are inconsistent with stated objectives, these findings point to the need for improved metrics when evaluating the quality of environmental decision processes.

Good decisions, bad decisions: the interaction of process and outcome in evaluations of decision quality

“…Most expert discussions of decision quality focus on the extent to which a given decision meets certain process-oriented criteria (National Research Council 2005;Wilson and Arvai 2006). However, the results reported here suggest that outside observers (namely members of 'the public') do not use process-based criteria to evaluate decision quality and rely on the suitability of outcomes instead.…”

“…The HQ process was described in terms of participatory and structured decisionmaking (SDM) akin to the process outlined by Keeney (1992) and others (von Winterfeldt and Edwards 1986;Hammond, Keeney, and Raiffa 1999;Gregory 2000;Gregory, Arvai, and McDaniels 2001;National Research Council 2005;Wilson and Arvai 2006), and later field tested in a variety of contexts (McDaniels, Gregory, and Fields 1999;Gregory and Wellman 2001;Arvai and Gregory 2003;Wilson and McDaniels 2007). Briefly, SDM processes are designed based on insights from behavioral decision research (Mellers, Schwartz, and Cooke 1998;Slovic and Lichtenstein 2006) and multiattribute utility theory (Keeney and Raiffa 1993), and are defined by engaging relevant stakeholders in the following tasks: (1) defining the decision to be made; (2) identifying stakeholders' objectives in context of the decision; (3) creating a set of alternatives that are sensitive to stakeholders' objectives to address the problem; (4) identifying consequences across objectives that are associated with each alternative; and (5) addressing the important tradeoffs that choosing one alternative over another entails.…”

“…We recognize that this is a controversial proposition (von Winterfeldt and Edwards 1986;Wilson 1991;Wilson and Arvai 2006) but, nevertheless, used it as the basis for our hypotheses. As far as trust is concerned, there are arguments that support both decision-making processes and outcomes as important drivers of how people evaluate the performance of risk managers (Renn and Levine 1991;Slovic 1999;Arvai 2003;Poortinga and Pidgeon 2003).…”

Grappling with the social dimensions of novel ecosystems

“…). Instead, methods to measure outcomes depend on site‐specific conditions and individual project parameters (Wilson and Arvai ). The novel ecosystem concept expands the potential suite of management objectives for modified ecosystems by removing limitations of conservative conservation strategies and increasing flexibility to work with the extant system (Hobbs et al .…”

Indicators of Informal and Formal Decision-making about a Socioscientific Issue

“…Others have noted the difference between individuals' initial -value-judgments‖ and reasoned values-based decisions. For example, Wilson and Arvai (2006) found that participants in an experiment were unable to stay focused on stated conservation values (objectives) that were intended for a formal decision-making process, and ultimately, participants made decisions that reflected their initial affective impressions that aligned with informal mental processes (Wilson, 2008). In contrast, formal decision-making is rooted in internal consistency between a suite of values and a reasoned analysis of tradeoffs instead of being rooted in affective impressions.…”