Going “high stakes” with a pharmacy OSCE: Lessons learned in the transition

A Process for Curricular Improvement Based on Evaluation of Student Performance on Milestone Examinations

“…However, over the last two years this strategy has evolved into identifying and improving area(s) of weakness, such that students are now required to remediate any component where they do not achieve a score of 70% or greater. This decision was made because increasing examination stakes (Table 1) has been shown to increase student performance, 5,7 and faculty members felt that it was in the best interest of students to ensure that they were prepared for APPEs and the North American Pharmacist Licensure Examination (NAPLEX). Currently, students who do not achieve passing scores on any OSCE/SPA components of the P1 and P2 milestone examinations remediate that component by completing quiz banks (with a specified passing score of 85%) on the related therapeutic topic(s) in Access Pharmacy (Top 200 Drugs Challenge) or the American Pharmacists Association (APhA) (APhA Complete Review for Pharmacy).…”

“…3 There are several reports on the use of progress examinations in pharmacy education, including MileMarker or milestone examinations. [4][5][6][7][8][9][10] These are cumulative and comprehensive examinations designed to assess student learning, knowledge retention, and in some instances professional competencies. The composition of the examinations varies from exclusively multiple-choice questions (eg, general knowledge and case-based questions) to those with performance-based assessments through objective structured clinical examination (OSCE)/standardized patient assessment (SPA) activities.…”

“…The composition of the examinations varies from exclusively multiple-choice questions (eg, general knowledge and case-based questions) to those with performance-based assessments through objective structured clinical examination (OSCE)/standardized patient assessment (SPA) activities. [4][5][6][7][8][9][10] Ideally, no information about the content of the examination is provided to students ahead of time as assessing student performance immediately following focused study is not the desired intent of these examinations. The advantage of this type of examination over course examinations is that they should provide a more accurate evaluation of the knowledge and competencies acquired and retained by students.…”

“…The stakes associated with progressive examinations have been shown to impact student performance with students performing better on higher stakes examinations. 5,7 Studies examining the effect of incentives on student performance have shown that negative incentives (eg, remediation, failure to progress) are more effective than positive incentives (eg, rewards such as books or bonus points). Additionally, examinations with negative high-stakes incentives (ie, failure to progress in the curriculum) are more effective than low-stakes incentives (ie, remediation).…”

“…Additionally, examinations with negative high-stakes incentives (ie, failure to progress in the curriculum) are more effective than low-stakes incentives (ie, remediation). 5 While there are several reports on the usefulness of progress examinations as a means for assessing student achievement of learning outcomes and clinical competencies in pharmacy education, [4][5][6][7][8][9][10] there are no reports on how the results from these examinations can be used for curricular improvements.…”