Quality of Feedback Using Workplace Based Assessments in Urological Training

Abstract

Introduction: The ISCP (Intercollegiate Surgical Curriculum Programme) was introduced in 2007. It heralded a shift in surgical training from the traditional apprenticeship model, to a system where competence is assessed regularly within the workplace. Central to ISCP is the recording of Workplace Based Assessments, such as PBA (procedure-based assessment) and DOPS (Direct Observation of Procedural Skills). These are assessments ''for'' learning rather than assessments ''of'' learning and central to their purpose is detailed feedback from trainer to trainee. The aim of this study was to assess the quality of the feedback recorded on ISCP. Methods: The anonymised ISCP portfolios for 170 core and specialty urology trainees working in 16 deaneries were analysed. Over a three month period 469 DOPS and 592 PBAs were recorded and the presence and quality of assessors' feedback were evaluated according to the presence of five criteria: • Encouragement through positive language, • Including strengths, • Raising appropriate development needs, • Suggestions for appropriate corrective action, • Providing explanations. Results: Feedback was recorded in 49% and 67% of PBAs and DOPS respectively. In those assessments where feedback was recorded, positive language was used in 83% in PBAs and 70% in DOPS. Strengths were recorded in 53% for both tools. Developmental needs, suggestions for development and detailed explanation were recorded in 25%, 26%, 29% for PBAs and 24%, 23%, 19% for DOPS respectively. Conclusion: This study highlights the lack of feedback being recorded on ISCP and the paucity of good quality feedback when it was given.

‘Playing the game’: How do surgical trainees seek feedback using workplace-based assessment?

“…Many trainees perceived WBA as an assessment of their learning and others were unclear about the purpose of WBA, as in other studies. Trainees may have difficulty in perceiving WBAs as being for learning as only half of WBAs contain written feedback, which may not identify specific areas for performance improvement …”

“…Trainees actively seek feedback after they believe they have performed well; this is also documented practice by trainees in difficulty . In practice, trainers must balance their roles as teacher and assessor, with evidence suggesting trainers’ reluctance to give negative feedback to trainees whilst maintaining an educational relationship . Therefore, WBAs may not currently facilitate a culture of learning and feedback within medicine, for which they were designed, but instead a culture of ‘gaming’.…”

“…Many trainees perceived WBA as an assessment of their learning 3,10,11,27 and others were unclear about the purpose of WBA, 28 as in other studies. Trainees may have difficulty in perceiving WBAs as being for learning as only half of WBAs contain written feedback, 12 which may not identify specific areas for performance improvement. 29 Trainees' perceptions of their trainers are entwined with their decision making around seeking feedback using WBA, and trainees' perceptions of their trainers' supportiveness, [1][2][3]30 clinical skills, 30 credibility 30,31 and engagement with WBA are all reported to play a role.…”

“…However, a recent review suggests that the educational impact of WBA remains unproven, 9 with others suggesting that feedback is not synonymous with WBA. [10][11][12] Research with Dutch GPs, where WBAs were formative and often voluntary, suggested that deliberate planning of observations and feedback was essential to undertaking WBA. However, apprehension about being observed and receiving feedback had a powerful negative effect on use of WBA.…”

Work-based assessments: making the transition from participation to engagement

“…Ali et al. 24 retrospectively analysed the feedback that had been recorded in the WBAs undertaken by urology trainees and discovered that there was a lack of feedback and when there was feedback it was of poor quality. Holmboe et al.…”