Non-technical skills assessments in undergraduate medical education: A focused BEME systematic review: BEME Guide No. 54

Abstract

Background Many medical schools have implemented curricula to teach non-technical skills, a personal set of complex social and cognitive skills which are grounded in human factors safety industries in and out of health. Consensus on how to assess these skills is lacking. This systematic review aimed to evaluate the evidence regarding non-technical skills assessments in undergraduate medical education, to describe the tools used, learning outcomes and the validity, reliability and psychometrics of the instruments. Given the discrete context, a focussed review model is being deployed. Methods Studies describing assessment methods as either the focus of the study or having non-technical skills assessment as an outcome measure of the research were considered. A standardized search of online databases was conducted and consensus reached on included studies. Data extraction, quality assessment and content analysis were conducted per Best Evidence in Medical Education guidelines. Results Nine papers met the inclusion criteria. Assessment methods broadly fell into three categories: simulated clinical scenarios, objective structured clinical examinations, and questionnaires or written assessments. Details of methodology were synthesised to support readers developing their own materials. Tools to assess non-technical skills were often developed locally, in 4 response to specific educational interventions, without reference to conceptual frameworks. Consequently, the tools were rarely validated, limiting dissemination and replication. The majority of studies achieved outcomes modifying knowledge and skills of participants. Two studies resulted in behavioural change and one resulted in change in practice. Conclusions There were clear themes in content and broad categories in methods of assessments employed, with the OSCE identified as most able to assess multiple related skills at once. The quality of this evidence was poor due to lack of theoretical underpinning, with most assessments not part of normal process, but rather produced as a specific outcome measure for a teaching based study. Data on validity, reliability and learning outcomes was not available so these questions cannot be addressed at this time. Whilst the current literature forms a good starting position for educators developing materials, there is a need for future work to address these weaknesses as such tools are required across health education.

Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63

“…The review team postulated, in line with previous BEME reviews, that this is partly related to the complexity of educational developments and therefore requires an approach that can address and account for this complexity. Thus, we considered two distinct quality elements: 1) the risk of bias or quality of the study design when outcomes were reported (similar to the Cochrane tradition) and 2) the risk of bias or quality of reporting for the educational development itself (as previously used by Gordon, Farnan, et al (2019) and Gordon et al (2018), originally modified from Reed et al (2005)). The latter is critically important, because only when the development is robustly described, can educators or researchers hope to replicate the results in other contexts.…”

Exploring transformative learning when developing medical students' non‐technical skills

“…This is in keeping with a recent Best Evidence in Medical Education (BEME) systematic review of NTS assessments in medical education, which noted that failure to incorporate NTS training within medical students' education could result in the undervaluing of the area by students. 9 The medical students in this study described a clear transition in their perspectives on NTS and the impact of such skills on patient care, and articulated how this change of perspective might influence their future behaviour (Phases 5 and 6).…”

“…In its Generic Professional Capabilities Framework, the UK General Medical Council outlines the necessity for doctors to demonstrate and apply NTS to improve the quality and safety of patient care . It is recognised that training in NTS should be incorporated into the curricula of medical students, but the lack of consensus or recommendations with reference to such training has resulted in significant heterogeneity in approaches to teaching such skills . Furthermore, the way in which such teaching interventions have been evaluated is also variable.…”

A Delphi consensus study for teaching “Basic Trauma Management” to third-year medical students

“…The influence of these human factors on clinical outcomes has already been ascertained [37, 38]. In the undergraduate setting, however, there is a substantial lack of guidance and teaching for these skills [39, 40]. Our stakeholders underlined the need to teach human factors, which might represent a trend that is already occurring in the postgraduate medical education milieu , as a shift towards a more holistic model of medical education.…”