The passing score in the Objective Structured Clinical Examination

Abstract

The objective structured clinical examination (OSCE) now has an established place in the assessment of the medical undergraduate. While much has been written about the reliability of the OSCE, empirical work on the determination of the passing score which represents competence on the OSCE is rarely encountered. If the OSCE is to play its role in the 'high stakes' testing of clinical competence, it is important that this passing score be set reliably and defensibly. This article illustrates how a two-session modified Angoff standard-setting procedure is used to set the passing score on a 14 station Obstetrics and Gynaecology OSCE used to assess final year students at The Queen's University of Belfast. The Angoff methodology harnesses the professional judgement of expert judges to establish defensible standards. Four university teachers, five non-academic consultants and six junior clinical staff took part in a two-session Angoff standard-setting procedure. In the first session, the judges (individually and in silence) used their professional judgement to estimate the score which a minimally competent final year obstetrics and gynaecology student should achieve on each tested element of the OSCE. In the second session they revised their session 1 judgements in the light of the OSCE scores of real students and the opportunity for structured discussion. The passing score for the OSCE is reported together with the statistical measures which assure its reliability.

Estimating the Minimum Number of Judges Required for Test-centred Standard Setting on Written Assessments. Do Discussion and Iteration have an Influence?

“…There have been many modifications to both methods, and the consensus of expert opinion (Norcini, 2003;Morrison et al, 1996) recommends that standard setting should be an iterative process, allowing for discussion between judges and potential review of standards in the light of discussion, together with feedback from performance data. Standard setting methods described here have been implemented with these recommendations in mind.…”

“…Previous authors have provided useful examples of standard setting in the medical school context (Morrison et al, 1996;Muijtjens et al, 1998;Verhoeven et al, 1999;Wilkinson et al, 2001;Downing et al, 2003). However the resource issue of the number of judges required to obtain a reliable standard for small-scale assessments has not been specifically addressed.…”

Evaluation of student’s competencies on Social Medicine internship with the Objective Structured Clinical Examination method

“…However, there is no ideal or standard method. (livingstone S and Zieky M, 1982;Newble D, 1994) In literature, Angoff method is the most widely used (Livingstone S and Zieky M, 1982;Morrison H, 1996). Each board member determines individually for each item of each question the expected percentage of "boundary candidates" or minimally competent who should answer correctly.…”

Reliability and Credibility of Progress Test Criteria Developed by Alumni, Faculty, and Mixed Alumni-Faculty Judge Panels

“…Providing judges with past item difficulties (item p values) would have established a realistic starting point for per-item performance of the overall student body and may have facilitated estimation of borderline student performance. 19,25,26 Revision routinely occurs after items are tested. We chose not to provide item difficulty levels to judges because revision of an item has the potential to change item difficulty, thus rendering past performance estimates invalid.…”