Objective. To evaluate the validity and reliability evidence of the preceptor assessment of student tool (PAST) which was designed to assess doctor of pharmacy (PharmD) student rotation performance. Methods. Evaluation data were loaded into WINSTEPS software to conduct a Rasch rating scale analysis. Validity evidence was examined from construct and content validity perspectives, and reliability was assessed via student and item separation index and reliability coefficient. Data from 435 observations were included in the analysis. Results. All 19 items measured the same construct of interest and the five-point rating scale functioned appropriately and differentiated students' ability. However, the item/person map indicated an absence of items at the end of the measurement continuum. Conclusion. Although adding items at the end of the measurement continuum may be beneficial, PAST showed good validity and reliability evidence when used to evaluate PharmD student rotations and is suitable to assess mastery learning.Keywords: doctor of pharmacy students, Rasch analysis, instrument evaluation, performance assessment, APPE/ IPPE
INTRODUCTIONAll doctor of pharmacy (PharmD) programs in the United States require students to gain experience in pharmacy practice through introductory level (IPPE) or advanced pharmacy practice experiences (APPE) before graduation.1,2 The overall goal of the various pharmacy practice experiences is to educate students to be competent health care professionals who will provide patients with optimal pharmaceutical care. This goal is the culmination of multiple entities and reports. Based on the Institute of Medicine (IOM) report on health professions education, health care providers should focus on five competences: delivering patient-centered care, using evidence-based practice, improving the care quality, employing information technology, and integrating with interdisciplinary teams.3 Additionally, the American Association of Colleges of Pharmacy's (AACP) Center for the Advancement of Pharmacy Education (CAPE) 2013 released a list of terminal educational outcomes for PharmD students that include four main domains (ie, Foundational Knowledge, Essentials for Practice and Care, Approach to Practice and Care, and Personal and Professional Development) with 15 subcategories. 4 To ensure that students and colleges are meeting CAPE outcomes and ACPE accreditation standards, frequent assessment of student knowledge, skills and abilities is required and the evaluation instrument used to assess students' performance should be reliable and valid.
5The College of Pharmacy at the University of Arizona developed a preceptor assessment of student tool (PAST), based on the CAPE 2013 competencies, aimed at assessing students' performance on rotations and measuring their mastery learning of the CAPE outcomes. Volunteer faculty, known as preceptors, used the instrument to evaluate students' performance and assigned them a final grade for each rotation. PAST comprises 19 items covering the CAPE competencies (Appe...