Abstract:This is the first nationwide study offering a deeper insight into factors influencing T&O trainees' perceptions of the educational benefit gained from using PBAs. This study informs the debate on how to improve the effective use of PBAs in T&O training, and generally, of workplace-based assessments in surgical training.
“…For example, one study on MSF reported that 89% of trainees agreed the comments were useful, whereas 38% agreed the numeric report was useful . Trainees particularly valued the verbal delivery of performance information . In a study conducted by Sabey and Harris, 94% of trainees described their verbal feedback as useful compared with 74% for written comments .…”
Section: Resultsmentioning
confidence: 99%
“…These perspectives led to a view of such assessments as ‘form‐filling’ or ‘tick‐box’ initiatives:Now it is just so people can tick the boxes. The inspiration and pride in our profession has gone.…”
Section: Resultsmentioning
confidence: 99%
“…Many studies showed that trainees influenced their assessment messages strategically. Those who did not just want ‘marks on a sheet of paper’ chose assessors they knew were serious about assessments, and whom they anticipated would give them quality feedback and take time to go through the assessment with them immediately afterwards . Others focused on ratings only or ‘gamed’ the system, seeking out assessors known to be lenient in their assessments .…”
Section: Resultsmentioning
confidence: 99%
“…33 Trainees particularly valued the verbal delivery of performance information. 34 In a study conducted by Sabey and Harris, 94% of trainees described their verbal feedback as useful compared with 74% for written comments. 20 This valuing of verbal feedback was especially marked when in-depth conversations occurred 19 and areas of weakness were identified.…”
Section: Trainees Value Developmental Assessment Messagesmentioning
confidence: 99%
“…For example, between 20% and 29% of trainees felt they improved their practice because of feedback, 34,38 with those who reported receiving good quality feedback being more likely to agree. 34 Some felt that the assessment messages, often coupled with dialogue, helped them to develop reflective skills 25,37,39 and self-assessment skills. 19 Trainees highlighted additional aspects of the assessment messages they valued, such as their breadth, including coverage of non-technical skills 18 or their provision of a more holistic picture of performance: 23,25,33 What I think is more useful is when you get feedback about how you're performing as a trainee in general.…”
Section: Trainees Value Developmental Assessment Messagesmentioning
Objectives
This study was designed as a narrative systematic literature review of medical specialist trainees’ perspectives of the assessment messages they receive in the context of clinical performance assessments. The aim of the study was to determine if trainees value the information they receive through the formats designed to promote their development and, if not, the reasons for this.
Methods
The authors searched the ERIC, EMBASE, Ovid MEDLINE and PsycINFO databases for articles published up to 16 June 2018 that present original data on trainees’ perspectives of the assessment messages they receive in the context of work‐based assessments (WBAs) and in‐training assessments (ITAs) used within their training programmes. All authors screened 938 abstracts and 139 full‐text articles were assessed after this. Descriptions of quantitative data and thematic analysis of qualitative data were used to present the opinions of trainees.
Results
Thirty‐three articles met the inclusion criteria. Twenty‐six articles (79%) described trainees’ perspectives in the context of WBA and the remaining articles referred to ITA formats. Wide‐ranging opinions were reported. The analysis categorised these into three themes: trainees value developmental assessment messages; trainees become disengaged when assessment messages are not developmental, and trainees’ views depend on the environment, the assessor and themselves. Some trainees reported that the assessment messages were valuable and provided input on their performance to guide their development, but many disagreed. In particular, the trainee's own level of engagement with the assessments influenced his or her perspectives on the messages received.
Conclusions
Trainees do not universally perceive that clinical performance assessments provide them with the valuable developmental input on their performance they were designed to do. Factors related to the environment, the assessor and themselves influence their perspectives.
“…For example, one study on MSF reported that 89% of trainees agreed the comments were useful, whereas 38% agreed the numeric report was useful . Trainees particularly valued the verbal delivery of performance information . In a study conducted by Sabey and Harris, 94% of trainees described their verbal feedback as useful compared with 74% for written comments .…”
Section: Resultsmentioning
confidence: 99%
“…These perspectives led to a view of such assessments as ‘form‐filling’ or ‘tick‐box’ initiatives:Now it is just so people can tick the boxes. The inspiration and pride in our profession has gone.…”
Section: Resultsmentioning
confidence: 99%
“…Many studies showed that trainees influenced their assessment messages strategically. Those who did not just want ‘marks on a sheet of paper’ chose assessors they knew were serious about assessments, and whom they anticipated would give them quality feedback and take time to go through the assessment with them immediately afterwards . Others focused on ratings only or ‘gamed’ the system, seeking out assessors known to be lenient in their assessments .…”
Section: Resultsmentioning
confidence: 99%
“…33 Trainees particularly valued the verbal delivery of performance information. 34 In a study conducted by Sabey and Harris, 94% of trainees described their verbal feedback as useful compared with 74% for written comments. 20 This valuing of verbal feedback was especially marked when in-depth conversations occurred 19 and areas of weakness were identified.…”
Section: Trainees Value Developmental Assessment Messagesmentioning
confidence: 99%
“…For example, between 20% and 29% of trainees felt they improved their practice because of feedback, 34,38 with those who reported receiving good quality feedback being more likely to agree. 34 Some felt that the assessment messages, often coupled with dialogue, helped them to develop reflective skills 25,37,39 and self-assessment skills. 19 Trainees highlighted additional aspects of the assessment messages they valued, such as their breadth, including coverage of non-technical skills 18 or their provision of a more holistic picture of performance: 23,25,33 What I think is more useful is when you get feedback about how you're performing as a trainee in general.…”
Section: Trainees Value Developmental Assessment Messagesmentioning
Objectives
This study was designed as a narrative systematic literature review of medical specialist trainees’ perspectives of the assessment messages they receive in the context of clinical performance assessments. The aim of the study was to determine if trainees value the information they receive through the formats designed to promote their development and, if not, the reasons for this.
Methods
The authors searched the ERIC, EMBASE, Ovid MEDLINE and PsycINFO databases for articles published up to 16 June 2018 that present original data on trainees’ perspectives of the assessment messages they receive in the context of work‐based assessments (WBAs) and in‐training assessments (ITAs) used within their training programmes. All authors screened 938 abstracts and 139 full‐text articles were assessed after this. Descriptions of quantitative data and thematic analysis of qualitative data were used to present the opinions of trainees.
Results
Thirty‐three articles met the inclusion criteria. Twenty‐six articles (79%) described trainees’ perspectives in the context of WBA and the remaining articles referred to ITA formats. Wide‐ranging opinions were reported. The analysis categorised these into three themes: trainees value developmental assessment messages; trainees become disengaged when assessment messages are not developmental, and trainees’ views depend on the environment, the assessor and themselves. Some trainees reported that the assessment messages were valuable and provided input on their performance to guide their development, but many disagreed. In particular, the trainee's own level of engagement with the assessments influenced his or her perspectives on the messages received.
Conclusions
Trainees do not universally perceive that clinical performance assessments provide them with the valuable developmental input on their performance they were designed to do. Factors related to the environment, the assessor and themselves influence their perspectives.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
