The Evolving Syntheses of Program Value

Abstract

In this paper, we object to Michael Scriven's claim that the basic logic of evaluation is criterial and standards-based. We note that valuing is an integral part of perception and that valuing within perception, repeatedly refined, is an even more basic logic of evaluation. We find unpersuasive his claim that making the final synthesis “governed” will diminish bias, noting that bias will find its way into the required statements of need, function, standards and weighting. We offer our alternative for disciplining the synthesis process, by urging more systematic and demanding critiques of emerging interpretations and values, and by more deliberately using competing conceptual organizers (e.g., goals, issues, decisions and elements of the rules Scriven advocates) as temporary and dialectical grounds for reconsidering the evolving meanings of the program, including its merit and shortcoming.

The overall evaluative conclusion as an element of evaluation practice

“…The final stage considers what is still puzzling and seeks to explore those puzzles rather than explain them away (Eoyang & Oakden, 2016). Stake et al (1997) also highlight the importance of looking for patterns. They advocate for "disciplined intuition", describing intuition as "pulling hidden patterns together so that we can make better judgments about the evaluand" (Stake et al, 1997, p. 96).…”

“…Stake et al (1997) reject criterial treatment of an evaluand in favour of a holistic approach that values experiential knowledge. The contrast between this holistic, experiential approach with the more common criterial or analytic approach is most evident in Scriven's (1994) paper entitled "The Final Synthesis", presenting the criterial approach, and Stake's response to it, written with a group of his graduate students (Stake et al, 1997). These papers are cited in other literature as presenting two important perspectives on valuing, and particularly on the drawing of overall evaluative conclusions (Gates, 2017;House & Howe, 1999;Julnes, 2012aJulnes, , 2012b.…”

“…This approach is most evident in the work of Stake, particularly his responsive evaluation (Stake, 2004). Stake et al (1997) reject criterial treatment of an evaluand in favour of a holistic approach that values experiential knowledge. The contrast between this holistic, experiential approach with the more common criterial or analytic approach is most evident in Scriven's (1994) paper entitled "The Final Synthesis", presenting the criterial approach, and Stake's response to it, written with a group of his graduate students (Stake et al, 1997).…”

“…These papers are cited in other literature as presenting two important perspectives on valuing, and particularly on the drawing of overall evaluative conclusions (Gates, 2017;House & Howe, 1999;Julnes, 2012aJulnes, , 2012b. Stake et al (1997) don't reject the view that evaluation is about judging the merit or worth of an evaluand, or that this culminates in a synthesis or overall evaluative conclusion. Indeed, they "applaud [Scriven's] effort to make final syntheses more carefully reasoned" (Stake et al, 1997, p. 90).…”

“…He defines synthesis as "the process of combining a set of ratings or performances on several dimensions or components into an overall rating" (Scriven, 1991, p. 342). Stake et al (1997) calls for a holistic approach to synthesis. He defines synthesis as "the act of putting together, assembling the parts; making sense of complex issues" (Stake, 2004, p. 160).…”