Why Get Involved in Program Evaluations?: Toward a Model of Stakeholder Involvement Motives

Abstract

It is generally recognized that for any evaluation to be successful there needs to be a significant amount of involvement from program staff and other stakeholders who are invested in the program. Ideally, these individuals would be highly engaged in the evaluation process; however, in practice this ideal is rarely met. Two research studies were undertaken to explore different stakeholder motives for involvement in program evaluations and their relative importance in involvement decisions. In Study 1, stakeholder views of their own involvement (or lack thereof) in recent program evaluations were analyzed qualitatively using a thematic analysis approach. The findings revealed diverse stakeholder motives for evaluation involvement that include personal/human factors (e.g., opportunity for personal advancement), evaluation factors (e.g., clarity of evaluation goals) and organizational factors (e.g., funding implications). A second study looked more closely at these motives to determine their relative importance to evaluation involvement decision-making. Using the Q-sort method, three involvement profiles were identified which revealed different pathways to involvement based on individual concerns for program funding, learning opportunities, and evaluation quality. A preliminary model of stakeholder involvement motives is proposed in which the value and relevance of the evaluation are determined by expected outcomes of the evaluation. Stakeholder Evaluation Involvement Motivesiii Dedication It has been said many times that all a child needs to succeed is one teacher who really cares about them and their future. In my life I have been blessed with such a gift three times over. And it is to them that this thesis is dedicated.To my sixth grade teacher, the late Mr. Don Kusyk… Thank you for teaching me how to love learning. I carried the lessons you taught and the confidence you had in me through three post-secondary degrees. I am sorry I did not get to tell you in person how much you meant to me, and how much of this paper I owe to you. To my parents, Casey and Marie Rocznik, two dedicated public school teachers…Thank you for instilling in me the value and worth of an education, for never questioning my educational choices, and for cheering me on through it all. I could not have -would not have -done any of this without your guidance, love, and support. Stakeholder Evaluation Involvement Motives iv AcknowledgementsIt is odd to call this an independent research project when its existence relied on so many other people. To all my family, friends, coworkers, and colleagues, thank you for your support, guidance, and friendship over the last seven years.Sincere and heartfelt thanks to my supervisor, Bernadette Campbell, for your tireless effort guiding, shaping, and editing this document. I so appreciate all the work you put in to help make this project the best it could be. I would like to thank all of my committee members, Isabelle Bourgeois, Connie Kristiansen, Karen Schwartz, and Warren Thorngate for their helpful questi...

Introducing Q Methodology to Program Evaluators

“…In one participatory evaluation (Lazard et al, 2011), program staff administered the Q-sorts to program participants, promoting process use. Evaluators have also used Q to explore stakeholder engagement in participatory evaluation in order to identify participants' initial motivations (Militello et al, 2016;Thompson, 2015), perceptions of the process (Cotton & Mahroos-Alsaiari, 2015;Danielson et al, 2009), and their methodological preferences (Kelly et al, 2016).…”