Assessing competency-based evaluation course impacts: A mixed methods case study

Poth, Cheryl; Searle, Michelle; Aquilina, Alexandra M.; Ge, Jenny; Elder, Alexa

doi:10.1016/j.evalprogplan.2020.101789

Cited by 17 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We adopted mixed-methods [34] as the best way to assess the impacts of the project "The School Grows with Us". The use of mixed methods in terms of evaluation allows one to understand the internal and external influences and the various elements that characterize and/or interfere with the process of change [35,36].…”

Section: Methodsmentioning

confidence: 99%

Upgrading a Learning Context: Evaluation of Sintra’s Educational Project

2020

View full text Add to dashboard Cite

The changes observed in the school context demand new practices and impose new challenges to the operational assistants that, due to their relevant role in the educational environment, must be prepared and endowed with knowledge and skills to conduct their profession in a fully useful way. This is only possible through the promotion of their training and capacitation in a real work context. Through the European project entitled “Innovative Plans to Combat School Failure” which was implemented in Portugal, we assess the impacts of a training-capacitation action directed to operational assistants and explore the dynamics and influences underlying the learning process put in practice in the schools of the county of Sintra. This assessment conducted by a higher education institution (Iscte-University Institute of Lisbon) mobilized a mixed methodology-survey and focus groups with operational assistants and interviews conducted to school directors. We verified that a training activity conducted in the real working context potentiates the performance of these professionals, namely in terms of autonomy and adaptation to different contexts and duties, conflict management and cooperation, whose effects reflected on the organizational dynamics of the school institutions of the county of Sintra.

show abstract

Section: Methodsmentioning

confidence: 99%

Upgrading a Learning Context: Evaluation of Sintra’s Educational Project

2020

View full text Add to dashboard Cite

show abstract

“…In the years since, this course has undergone several iterations. During implementation of the fall 2018 course, we undertook a formal examination of the experiences that learners attributed to their self-reported competency gains (for full study description see Poth et al, 2020). In the published mixed-methods study, we discussed seven novel insights generated by a systems-informed case-study methodology.…”

Section: Contexts Surrounding the Development And Implementation Of Tmentioning

confidence: 99%

“…Th is involves the instructor initially identifying the course outcomes, then determining acceptable course evidence, and then designing course experiences to support learners' development of the competencies outlined as the course outcomes. For the 2018 course iteration studied in the published report (Poth et al, 2020), the instructor identifi ed 15 competencies from the initial framework CES (2018 ) as the course outcomes, four summative assessments as the course evidence (evaluation design, issue examination, discussion facilitation, and competency assessment and refl ection), and four types of feedback-embedded instructional activities occurring in person and as the course experiences (large group discussions about community partnership development, small group review of work-in-progress for the evaluation design, instructor facilitation of real-world evaluation skill applications, and engagement with diverse perspectives through readings, guest speakers, and video links).…”

Section: Describing the Course Design And Implementation Comprising Tmentioning

confidence: 99%

Competency-Based Evaluation Education: Four Essential Things to Know and Do

Poth

Searle

2021

Canadian Journal of Program Evaluation

Self Cite

View full text Add to dashboard Cite

Evaluator education must provide robust opportunities to support and assess the progressive, lifelong development of relevant knowledge and skills. If we wish to keep pace with the increasingly complex contexts in which evaluators operate, we need to better align our educational approaches with the global movement towards practice competencies guiding the profession. Key among the challenges is the lack of instructional guidance specific to a competency-based approach to evaluator education. In this practice note, we orient readers to the value of competency-based evaluation education and describe the teaching context using a systems perspective to examine the dynamic learning interactions and experiences. We advance four essential instructional features of the competency-based approach revealed by a study documenting the impacts on learning and student experiences. We conclude with lessons learned from reflecting upon our experiences during the development and implementation of a competency-based doctoral-level evaluation course to highlight the mutual benefits for learners and instructors.

show abstract

“…A career in evaluation involves working with stakeholders across multi-disciplinary backgrounds and contexts, and it attracts a diverse range of people with unique skill sets. As a result of this varied workforce, there has been a global focus on developing evaluator competencies and determining skillsets since the late 1980s (Diaz et al, 2020; Poth et al, 2020). Despite this, there has been limited research focussing on formal evaluation education (King & Ayoo, 2020); therefore, this article aims to explore the presence of formal evaluation programs in the current Australian higher education landscape.…”

Section: Introductionmentioning

confidence: 99%

Scoping recent investment in evaluation education in Australian universities

Quadrelli

Staheli

Cook-Watkins

et al. 2022

Evaluation Journal of Australasia

View full text Add to dashboard Cite

This article explores the current evaluation education program opportunities in the Australian higher education market and identifies potential research areas to inform pathways for future investment. Findings from our initial scoping phase are provided as a prelude to future work required in this space. The overarching research question investigates whether there is an opportunity to deliver further evaluation education programs in Australian higher education. Emerging from this question are three objectives: (a) to determine the current landscape of Australian university evaluation education program offerings; (b) to gain an understanding of the evaluation education discourse articulated within six selected evaluation journals and (c) to consider whether evaluation is a viable and meaningful discipline within higher education programs. An action research approach was adopted commencing with a benchmarking activity of Australian university offerings, followed by a bibliometric analysis of six evaluation journals, and concluding with a rapid scan of the literature. The selected methodologies reveal a potential gap in the higher education market, with the literature identifying inconsistent and mismatched programs while emphasising the need to invest in evaluation education. This article seeks to stimulate debate about formal higher education qualifications in evaluation and highlights the importance of potential future curriculum structure that can offer evaluators value, utility and growth.

show abstract

Assessing competency-based evaluation course impacts: A mixed methods case study

Cited by 17 publications

References 31 publications

Upgrading a Learning Context: Evaluation of Sintra’s Educational Project

Upgrading a Learning Context: Evaluation of Sintra’s Educational Project

Competency-Based Evaluation Education: Four Essential Things to Know and Do

Scoping recent investment in evaluation education in Australian universities

Contact Info

Product

Resources

About