Evaluating the Application of Decision Analysis Methods in Simulated Alternatives Assessment Case Studies: Potential Benefits and Challenges of Using MCDA

Abstract

We compare how several forms of multicriteria decision analysis (MCDA) can enhance the practice of alternatives assessment (AA). We report on a workshop in which 12 practitioners from US corporations, government agencies, NGOs, and consulting organizations applied different MCDA techniques to 3 AA case studies to understand how they improved the decision process. Participants were asked to select a preferred alternative in each case using a different decision analysis approach: their unaided decision-making method, individual or lightly facilitated group multiattribute value theory (MAVT), and more extensively facilitated group structured decision making (SDM). Surveys conducted after each exercise revealed that participants were positive toward the use of formal decision-making methods for AA, reporting meaningful increases in their understanding of the trade-offs involved and their own values. Participants also reported challenges with each approach. While the MCDA techniques were reported to enhance transparency and communication, they did not consistently lead to higher satisfaction with a decision and/or outcome, and they were not more likely to be adopted within their organizations than unaided approaches. More formal decision-making methods have promise in the context of AA, but practitioners will need more guidance to use such tools successfully. Practitioners will also need to define what "success" constitutes; different approaches may be called for depending on whether the objective is increased understanding, satisfaction with the outcome, satisfaction with the process, or something else.

Lessons from the 2018 International Symposium on Alternatives Assessment: Advances and Reflections on Practice and Ongoing Needs to Build the Field

“…For example, Dubrow and Risotto () outlined the development of a more user‐friendly multicriteria decision analysis (MCDA) tool to assist companies and authorities with navigating trade‐offs in the analysis. However, researchers have found that formal decision support tools do not always lead to increased user satisfaction in a chosen alternative, suggesting the need for more guidance on how to use techniques such as MCDA (Malloy, Zaunbrecher, Batteate et al ) and structured decision making (Wilson and McDaniels ) in alternatives assessment processes (Beaudrie et al ).…”

“…Second, there was broad agreement on the important role of transparency in the alternatives assessment process, given that decision making occurs throughout the assessment process, not just in the final selection of alternatives (Malloy, Zaunbrecher, Batteate et al ; Zhou ; Heine and Nestler ). There is also a need for research and guidance on when and how to use different decision‐making approaches, tools, and methods to best fit the specific alternatives assessment context (Beaudrie et al ). Malloy, Zaunbrecher, Batteate et al () cautioned that decision tools cannot replace deliberation—people, not computers, need to make decisions; decision tools are a support method and should not provide cover for strategic behavior.…”

Towards Objectification of Multi-Criteria Assessments: a Comparative Study on MCDA Methods

“…Most research has focused on developing and improving new MCDA methods. They differ in many aspects, such as different techniques for determining the weights of criteria in the calculations, the complexity of the algorithms, the way preferences and evaluation criteria are represented, the type of data aggregation and the possibility of considering uncertain data [3].…”

Combining In Silico Tools with Multicriteria Analysis for Alternatives Assessment of Hazardous Chemicals: Accounting for the Transformation Products of decaBDE and Its Alternatives

“…The use of MCDA methods has become increasingly popular in the field of alternatives assessment, as these methods can be used not only for drawing more trustworthy conclusions but also for identifying the most critical criteria. 25 …”

“…The use of MCDA methods has become increasingly popular in the field of alternatives assessment, as these methods can be used not only for drawing more trustworthy conclusions but also for identifying the most critical criteria. 25 In our previous study, 26 an efficient hazard ranking tool was established for alternatives assessment by combining opensource quantitative structure−activity relationship (QSAR) model hazard data and MCDA methods with the consideration of data uncertainties. Herein, we extend this hazard ranking tool to include and select important transformation products predicted by in silico tools and use multiple MCDA methods for a joint hazard ranking.…”