A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees

Abstract

BackgroundWorkplace-based assessments were designed to facilitate observation and structure feedback on the performance of trainees in real-time clinical settings and scenarios. Research in workplace-based assessments has primarily centred on understanding psychometric qualities and performance improvement impacts of trainees generally.An area that is far less understood is the use of workplace-based assessments for trainees who may not be performing at expected or desired standards, referred to within the literature as trainees ‘in difficulty’ or ‘underperforming’. In healthcare systems that increasingly depend on service provided by junior doctors, early detection (and remediation) of poor performance is essential. However, barriers to successful implementation of workplace-based assessments (WBAs) in this context include a misunderstanding of the use and purpose of these formative assessment tools.This review aims to explore the impact - or effectiveness - of workplace-based assessment on the identification of poor performance and to determine those conditions that support and enable detection, i.e. whether by routine or targeted use where poor performance is suspected. The review also aims to explore what effect (if any) the use of WBA may have on remediation or on changing clinical practice. The personal impact of the detection of poor performance on trainees and/or trainers may also be explored.Methods/designUsing BEME (Best Evidence in Medical Education) Collaboration review guidelines, nine databases will be searched for English-language records. Studies examining interventions for workplace-based assessment either routinely or in relation to poor performance will be included. Independent agreement (kappa .80) will be achieved using a randomly selected set of records prior to commencement of screening and data extraction using a BEME coding sheet modified as applicable (Buckley et al., Med Teach 31:282-98, 2009) as this has been used in previous WBA systematic reviews (Miller and Archer, BMJ doi:10.1136/bmj.c5064, 2010) allowing for more rigorous comparisons with the published literature. Educational outcomes will be evaluated using Kirkpatrick’s framework of educational outcomes using Barr’s adaptations (Barr et al., Evaluations of interprofessional education; a United Kingdom review of health and social care, 2000) for medical education research.DiscussionOur study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education.Systematic review registrationThe review has been registered by the BEME Collaboration www.bemecollaboration.org.

A Systematic Review of Training in Symptom Management in Palliative Care Within Postgraduate Medical Curriculums

“…Furthermore, it has been previously applied in BEME systematic reviews enabling comparison with other educational reviews. 34,35 Tool items consider the study research question, subjects, data collection methods, completeness of the data, control…”

Examining the readiness of best evidence in medical education guides for integration into educational practice: A meta-synthesis

“…We identified 42 Guides published in Medical Teacher between 1999 and 2017. Thirty-six Guides were described as systematic reviews [ 14 – 49 ], three as reviews [ 50 – 52 ], two as realist reviews [ 53 , 54 ], and one as a review with quality grading of articles [ 55 ]. Most authors rationalized conducting their Guides due to a lack of knowledge synthesis covering their specific topic.…”

The economic cost of failure in clinical education: a multi-perspective analysis

“…in contexts in which lower scores are required to achieve a passing outcome) will identify a smaller volume of failure incidents; however, the clinical performance of students who fail will be lower, which is likely to result in a higher cost of failure per student. There is ongoing debate around the reliability and validity of many forms of workplace‐based assessment and their function as formative or summative assessments . In our study, the failure threshold was set at 50% using the Assessment of Physiotherapy Practice (APP), as rated on one occasion at the end of the clinical unit (but based on observed student performances throughout the clinical placement).…”