Professionalization and evaluation: A European analysis in the digital era

Abstract

It is expected that the number of evaluators will continue to grow in the near future. However, the heterogeneity of different national contexts makes the consolidation of a consistent ‘jurisdiction’ for the professional evaluator rather problematic. This article contributes to the debate on the professionalization of evaluators by looking at practices attributed, competences and skills required by employers, and the main topics addressed by the community of evaluators. The authors draw on various sources – ISCO08 (International Standard Classification of Occupation); ESCO (European Skills, Competences, Qualifications and Occupations); job offers posted on the EES (European Evaluation Society) website; EES LinkedIn group – to argue that the practice of evaluation has achieved a supranational dimension, with potential consequences both on evaluators’ educational profile and on the ways in which evaluations are commissioned and conducted.

So What Do You Do? Exploring Evaluator Descriptions of Their Work

“…Yet in this task, evaluation has stalled, frequently finding itself standing in competition with, not distinct from, other fields such as auditing or quality assurance (Beywl, 2006). This view finds empirical support in research conducted by Castro, Fragapane, and Rinaldi (2016) as well as recent analysis of evaluation job postings from the EES, which suggest there is no clear “evaluation jurisdiction.” Skills and competencies sought by evaluation employers overlapped considerably with those sought from other, similar job profiles, a finding that led Castro and colleagues (2016) to suggest the most serious hindrance to full professionalization in evaluation is the absence of a strong and consistent identity.…”

“…This lack of recognition is problematic for a number of reasons. First and foremost, evaluation is a discipline debating the merits of formal professionalization, that is, the process a field goes through to become a profession (Castro et al, 2016; Conner & Dickman, 1979; Gauthier et al, 2010; House, 1993; Jacob, 2008; Jacob & Boisvert, 2010; Montrosse-Moorhead & Griffith, 2017; Morell & Flaherty, 1978; Picciotto, 2011), with many practitioners and academics arguing in favor of such professionalization (Bickman, 1997; Picciotto, 2011). Supporters of this view suggest professionalization of the field would guide training, provide special privileges (e.g., access, salaries), enhance prestige, and offer a degree of respectability that is not possible without formal recognition (Becker, 1970; Larson, 1977; Picciotto, 2011).…”

“…A recurring complaint among evaluators is that the general public does not understand their work. Anecdotes to this effect are evident in writings on evaluation practice (King, 2003; Sturges, 2014) and professional commentary (Castro, Fragapane, & Rinaldi, 2016; Gargani, 2016). Experienced evaluator and academic Jean King (2003), for example, notes:It is awkward to be a program evaluator.…”

“…(p. 57)Beyond anecdotes, initial research on evaluator identity also suggests evaluators struggle to explain their work to those outside the field (LaVelle, 2011; Sturges, 2014). Recent analyses of European Evaluation Society (EES) LinkedIn posts reveal a persistent view among evaluators that we need to better define and distinguish our work (Castro et al, 2016). Earlier qualitative studies indicate similar findings: “It’s interesting when you’re talking to people who are outside evaluation because nobody ever knows what you’re talking about.…”

What Is Evaluation?: Perspectives of How Evaluation Differs (or Not) From Research

“…Perspectives of how evaluation differs (or not) from research Evaluators, emerging and experienced alike, lament how difficult it is to communicate what evaluation is to non-evaluators (LaVelle, 2011;Mason & Hunt, 2018). This difficulty in communicating what evaluation is stems partly from the field of evaluation having identity issues (Castro, Fragapane, & Rinaldi, 2016), leading to difficulty coming to a consensus of the definition of evaluation (Levin-Rozalis, 2003). Furthermore, the similarity between related fields-auditing, management consulting, and especially social science research-exacerbates the issue of defining and communicating about evaluation.…”