Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles

Abstract

Introduction: In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment. Methods: For fifteen programmes in health professions education affiliated with members of an expert group (n ¼ 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research. Results and discussion: Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes.

Ottawa 2020 consensus statements for programmatic assessment – 2. Implementation and practice

“…The sample included undergraduate and postgraduate programmes from six different countries across three continents. Following the Ottawa 2020 conference, the participants in the group who achieved consensus on the 12 principles of programmatic assessment (Heeneman et al 2021), plus one additional programme, were invited to complete a 16-item questionnaire about implementation of programmatic assessment.…”

“…At the 2020 Ottawa conference, consensus was reached on twelve theoretical principles of programmatic assessment (Heeneman et al 2021). The consensus group, recognising the challenges of implementation (Norcini and Burch 2007;Norcini et al 2018), identified that, in order for programmatic assessment and its principles to exert their educational benefits, it is important to share a common understanding of how these principles are implemented across different contexts and regulatory settings.…”

Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis

“…Our findings add to the existing debates about programmatic assessment in health professions education by suggesting that there are important and often neglected causal mechanisms that impact the students varied experiences during the implementation of programmatic assessment. This adds to the existing literature on implementation approaches involving complex, dynamic and multilevel systems 12,13 . Our findings add to the overarching causal mechanisms that can explain student experiences of programmatic assessment 12,13,56 .…”

“…The underlying theory and principles of programmatic assessment have been described in detail in the literature 7–11 . Notwithstanding, there is limited understanding of implementation approaches taken across different contexts that involve complex, dynamic and multilevel systems 12,13 . Few studies provide empirical data supported by theoretically informed explanations of how programmatic assessment is working, for whom and in what context? 14–16 This pragmatic approach is used in critical realist and realist evaluation, 17 contrasting with traditional approaches to assessment research that typically asks, “what works?” 18 There is thus a need to further develop appropriate research methodologies to ensure researchers are asking the appropriate questions when considering the impact of a complex educational intervention such as programmatic assessment 19 .…”

Surgery goes EPA (Entrustable Professional Activity) – how a strikingly easy to use app revolutionizes assessments of clinical skills in surgical training

“…The Swiss College of Surgeons intend to implement their new EPA-based core curriculum with the Surg-prEPAred app. Since the use of mobile technology for documenting WBA’s is also part of the newest consensus statements on assessment in medical education [ 19 , 20 ], several specialties in Switzerland are now using prEPAred to gain experience in working with EPAs. Several studies on this topic are on the way.…”