Development and Validation Processes for an Objective Structured Clinical Examination (OSCE) for Entry-to-Practice Certification in Pharmacy: The Canadian Experience

Abstract

Objectives. An objective structured clinical examination (OSCE) was developed and validated as an addition to the entry-to-practice examination for pharmacists that previously consisted of both a multiple-choice, case-based written test of clinical knowledge and a performance assessment. Methods. Designing the OSCE for entry-to-practice certification of pharmacists in Canada required extensive consultation with stakeholders, development of an examination blueprint outlining competencies to be assessed, careful development and validation of multiple OSCE stations with tasks linked to the blueprint, development of assessment instruments, field testing of stations, development and validation of standard-setting procedures, and protocols for data collection and analysis. Results. The examination was field tested, finalized, and first delivered in May 2001. Preliminary analysis of results indicated the development and validation processes were successful in producing an OSCE model that is reliable with valid outcomes, defensible, and feasible. Conclusion. The Pharmacy Examining Board of Canada's Qualifying Examination (Part II -OSCE)represents the first time that a multi-site national licensing exam for entry-level practitioners in pharmacy has incorporated an objective structured clinical examination component. Together, the written and OSCE exams provide a broad assessment of competency, to ensure entry-level practitioners meet standards of practice for the protection of the public.Keywords: Examination, Certification, Clinical Assessment, OSCE, Pharmacy Education, Test Format, Testing specific medical conditions or drug-related problems), or "standardized clients" (actors or other health professionals who have been specially trained to portray allied health professionals in an interdisciplinary health care context). A candidate in an interactive station is observed and assessed by a trained examiner using a standardized marking key. Non-interactive (or quiet) stations typically are written responses to tasks or problems and involve no direct observation and assessment.

Perception of Pharmacy Students in Malaysia on the Use of Objective Structured Clinical Examinations to Evaluate Competence

“…Moreover, our consultations during the development process led to our decision to allocate 15 minutes per station because similar tasks were given less or equal time limits elsewhere. 2,4,6,10,20 Standard setting procedures such as Angoff and borderline were not utilized in the examination, since ours was a pilot study. In the future we hope to use such methods for grading performance and testing validity.…”

“…1,[3][4][5] In fact, the OSCE has been proven and rated as the most reliable and valid tool for assessing clinical competency. [6][7][8] The Kulliyyah (ie, the Faculty) of Pharmacy at the International Islamic University Malaysia (IIUM) has taken a pioneering role during the 2005-2006 academic session by applying and evaluating the effectiveness of this assessment instrument among the fourth-year students for the final examination in clinical pharmacy. This method was introduced to our undergraduate program in the context of a desire to improve the quality of our evaluation techniques and ever-changing curriculum, and to be abridged of current instructional technologies in pharmaceutical education.…”

Validation and Evaluation of an OSCE in Undergraduate Doctor of Pharmacy Program

“…22,23 Although a larger number of stations has been used to increase the reliability of OSCE in a highstake setting (such as The Pharmacy Examining Board of Canada's Qualifying Examination), 14 a small OSCE station number, in conjunction with the written exam, can serve as a reliable, valid and economically feasible global assessment method of student performance. 18,20 Although we did not conduct any interrater reliability testing, the similarity in the exam scores indicated that the interrater and interpatient variability didn't affect reliability of the exam.…”

An Assessment Program Using Standardized Clients to Determine Student Readiness for Clinical Practice

“…7,[10][11][12] Colleges and schools of pharmacy have adapted and refined the innovation for use in performance-based assessment of pharmacy students. 13,14 The more inclusive term standardized client was chosen for the Kansas program to accommodate a broader set of assessment scenarios. By the early 2000s, US and Canadian medical and Canadian pharmacy license examinations required candidates to complete simulated live clinical encounters.…”