British surgeons' experiences of mandatory online workplace-based assessment

Abstract

SummaryObjectives An online workplace-based assessment tool, the Intercollegiate Surgical Curriculum Programme (ISCP), has become mandatory for all British surgical trainees appointed since August 2007. A compulsory £125 annual trainee fee has also been introduced to fund its running costs.The study sought to evaluate user satisfaction with the ISCP.Design and setting A total of 539 users across all surgical specialties (including 122 surgeons acting as assessors) were surveyed in late 2008 by online questionnaire regarding their experiences with the ISCP.Results Sixty-seven percent had used the tool for at least one year. It was rated above average by only 6% for its registration process and only 11% for recording meetings and objectives. Forty-nine percent described its online assessments as poor or very poor, only 9% considering them good or very good. Seventy-nine percent rated the website's user friendliness as average or worse, as did 72% its peer-assessment tool and 61% its logbook of procedures. Seventy-six percent of respondents had carried out paper assessments due to difficulties using the website. Six percent stated that the ISCP had impacted negatively on their training opportunities, 41% reporting a negative impact overall upon their training; only 6% reported a positive impact. Ninety-four percent did not consider the trainee fee good value, only 2% believing it should be paid by the trainee. ConclusionsThe performance of the ISCP leaves large numbers of British surgeons unsatisfied. Its assessments lack appropriate evidence of validity and its introduction has been problematic. With reducing training hours, the increased online bureaucratic burden exacerbates low morale of trainees and trainers, adversely impacting potentially upon both competency and productivity.

Intercollegiate Surgical Curriculum Programme: Where are We Now?

“…A previous survey illustrated that the majority of subjects were dissatisfied overall with ISCP use. 1 Our survey showed 51.3% of subjects reported dissatisfaction (Figure 3) with the ISCP. We did not attempt to stratify this into the workbased components of the ISCP in our survey, as this may have detracted from overall satisfaction because there will always be variability between the tastes of those using the website.…”

“…1 It had been anticipated that independent surveys and feedback at a regional and national level would prompt joint improvement and changes to the ISCP as a whole. While some changes have occurred, we wanted to ascertain whether high levels of satisfaction were now present for various aspects of the ISCP and how it is used both by trainees and trainers.…”

Opinions of the ePortfolio and workplace-based assessments: a survey of core medical trainees and their supervisors

“…A large study of 539 surgical trainees showed that nearly two-thirds felt that their online portfolio impacted negatively on their training opportunities because of the time required to complete assessments. 14 Similarly, a sample of GP trainees cited the lack of suffi cient protected time as a reason for poor compliance with the ePortfolio. As our results demonstrate, the trainers also struggle with time constraints, with 70% claiming diffi culties in fi tting the required amount of assessments for their trainees into their daily clinical schedule, a fi nding supported by other groups.…”

Competency based surgical training

“…There is evidence to suggest that senior evaluators and experienced assessors may give lower but more accurate ratings on a trainee's workplace performance. 42 Literature search on WBAs has shown that there is a perception of low effectiveness amongst trainers, 43 increased administrative burden on trainers and trainees, 44 lack of validity 45 and the creation of a 'tick box' mentality. 46 A study conducted by McKavanagh et al 30 have shown that consultants input into junior doctors' WBAs had been relatively poor, with 71% of the trainee doctors in agreement that their consultant supervisors were not very enthusiastic on completing their WBAs.…”