Improving assessment practice through cross-institutional collaboration: An exercise on the use of OSCEs

Malau‐Aduli, Bunmi S.; Teague, Peta-Ann; Turner, Richard; Holman, Benjamin W.B.; D'Souza, Karen; Garne, David L; Heal, Clare; Heggarty, Paula; Hudson, Judith N; Wilson, Ian G; Vleuten, Cees van der

doi:10.3109/0142159x.2015.1016487

Cited by 8 publications

(7 citation statements)

References 21 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We focused on exit‐level OSCEs of Australian medical schools given the requirement for common curriculum/assessment mapping to agreed national graduate outcomes, determined by the Australian Medical Council 59 . Participating schools are members of the Australian Collaboration for Clinical Assessment in Medicine (ACCLAiM) and have similar integrated, outcomes‐based curricula and similar OSCE processes, including some shared stations, examiner calibration exercises and similar scoring sheets consisting of a checklist and a global rating scale, with explicit performance category descriptors 60,61 . This study was approved by the James Cook University Human Research Ethics Committee (H6833) and accepted by all participating universities.…”

Section: Methodsmentioning

confidence: 99%

Examiners’ decision‐making processes in observation‐based clinical examinations

et al. 2020

Self Cite

View full text Add to dashboard Cite

Background Objective structured clinical examinations (OSCEs) are commonly used to assess the clinical skills of health professional students. Examiner judgement is one acknowledged source of variation in candidate marks. This paper reports an exploration of examiner decision making to better characterise the cognitive processes and workload associated with making judgements of clinical performance in exit‐level OSCEs. Methods Fifty‐five examiners for exit‐level OSCEs at five Australian medical schools completed a NASA Task Load Index (TLX) measure of cognitive load and participated in focus group interviews immediately after the OSCE session. Discussions focused on how decisions were made for borderline and clear pass candidates. Interviews were transcribed, coded and thematically analysed. NASA TLX results were quantitatively analysed. Results Examiners self‐reported higher cognitive workload levels when assessing a borderline candidate in comparison with a clear pass candidate. Further analysis revealed five major themes considered by examiners when marking candidate performance in an OSCE: (a) use of marking criteria as a source of reassurance; (b) difficulty adhering to the marking sheet under certain conditions; (c) demeanour of candidates; (d) patient safety, and (e) calibration using a mental construct of the 'mythical [prototypical] intern'. Examiners demonstrated particularly higher mental demand when assessing borderline compared to clear pass candidates. Conclusions Examiners demonstrate that judging candidate performance is a complex, cognitively difficult task, particularly when performance is of borderline or lower standard. At programme exit level, examiners intuitively want to rate candidates against a construct of a prototypical graduate when marking criteria appear not to describe both what and how a passing candidate should demonstrate when completing clinical tasks. This construct should be shared, agreed upon and aligned with marking criteria to best guide examiner training and calibration. Achieving this integration may improve the accuracy and consistency of examiner judgements and reduce cognitive workload.

show abstract

Section: Methodsmentioning

confidence: 99%

Examiners’ decision‐making processes in observation‐based clinical examinations

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This study is part of a larger project that explored the experiences of all stakeholders hosting or attending virtual OSCEs (vOSCEs) at medical schools that are affiliated to the Australasian Collaboration for Clinical Assessment in Medicine (ACCLAiM). ACCLAiM is comprised of 15 medical schools across Australia and New Zealand and fosters a community of practice centered around quality in clinical assessment (22)(23)(24).…”

Section: Study Contextmentioning

confidence: 99%

Has the OSCE Met Its Final Demise? Rebalancing Clinical Assessment Approaches in the Peri-Pandemic World

2022

View full text Add to dashboard Cite

The Objective Structured Clinical Examination (OSCE) has been traditionally viewed as a highly valued tool for assessing clinical competence in health professions education. However, as the OSCE typically consists of a large-scale, face-to-face assessment activity, it has been variably criticized over recent years due to the extensive resourcing and relative expense required for delivery. Importantly, due to COVID-pandemic conditions and necessary health guidelines in 2020 and 2021, logistical issues inherent with OSCE delivery were exacerbated for many institutions across the globe. As a result, alternative clinical assessment strategies were employed to gather assessment datapoints to guide decision-making regarding student progression. Now, as communities learn to “live with COVID”, health professions educators have the opportunity to consider what weight should be placed on the OSCE as a tool for clinical assessment in the peri-pandemic world. In order to elucidate this timely clinical assessment issue, this qualitative study utilized focus group discussions to explore the perceptions of 23 clinical assessment stakeholders (examiners, students, simulated patients and administrators) in relation to the future role of the traditional OSCE. Thematic analysis of the FG transcripts revealed four major themes in relation to participants' views on the future of the OSCE vis-a-vis other clinical assessments in this peri-pandemic climate. The identified themes are (a) enduring value of the OSCE; (b) OSCE tensions; (c) educational impact; and (d) the importance of programs of assessment. It is clear that the OSCE continues to play a role in clinical assessments due to its perceived fairness, standardization and ability to yield robust results. However, recent experiences have resulted in a diminishing and refining of its role alongside workplace-based assessments in the new, peri-pandemic programs of assessment. Future programs of assessment should consider the strategic positioning of the OSCE within the context of utilizing a range of tools when determining students' clinical competence.

show abstract

“…Data generated from the OSCE can be used to benchmark student attainment of core clinical competencies in a way that usefully informs schools about the efficacy of their teaching and assessment. The Australian Collaboration for Clinical Assessment in Medicine (ACCLAiM) is responsible for benchmarking exit‐year medical student assessment using OSCEs 1 . As a national benchmarking consortium, involving membership of fourteen medical schools across Australia, ACCLAiM is faced with the task of ensuring that benchmarked OSCE cases are being conducted to an agreed standard at participating schools to certify the validity and reliability of the student performance data generated.…”

Section: What Problems Were Addressed?mentioning

confidence: 99%