The Integrative Themes in Physiology (ITIP) project was a National Science Foundation-funded collaboration between the American Physiological Society (APS) and the Human Anatomy and Physiology Society (HAPS). The project goal was to create instructional resources that emphasized active learning in undergraduate anatomy and physiology classrooms. The resources (activity modules and professional development) addressed two factors thought to be limiting science education reform: instructors' knowledge of how to implement active learning instruction and time to design innovative curricula. Volunteer instructors with a strong interest in using active learning in their classrooms were recruited to use the ITIP modules and provide ease-of-use feedback and student assessment data. As the study unfolded, instructor attrition was higher than had been anticipated, with 17 of 36 instructors withdrawing. More surprisingly, instructors remaining with the project failed to use the modules and reported specific obstacles that precluded module use, including lack of support from academic leadership, unplanned class size increases and heavy teaching loads, a union strike, insufficient time to develop a mindset for change, inadequate technology/funding, an adverse human subjects ruling, incompatibility of modules with instructors' established content and expectations, and personal factors. Despite the lack of module use and obstacles, 8 of 19 site testers began independently to introduce new active learning instruction into their classrooms. In the larger picture, however, it is important to note that only 8 of the initial 36 volunteers (22%) actually ended up changing their instruction to include opportunities for student active learning. These findings underscore the difficulty of implementing instructional change in college classrooms.
IntroductionCompetency-based assessment of resident performance requires faculty who are prepared to provide these assessments. Studies have shown that end-of-rotation evaluations completed by faculty lack reliability and fail to identify important deficiencies in residents' performance. 1,2A number of factors account for the less-than-acceptable reliability, including variations in clinical context or patient complexity, 3 lack of longitudinal experiences with residents and patients, 4 and rater error. Types of rater error include leniency error, in which residents' abilities are rated higher than performance merits; halo or reverse halo, when performance in 1 domain affects ratings in other domains; range restriction, in which ratings fall within a narrow range; and error of undifferentiation, in which raters fail to differentiate among different performance domains. [5][6][7][8] In 2009 a Milestone Task Force of the Accreditation Council for Graduate Medical Education and American Board of Internal Medicine (ABIM) published its draft Milestone document, consisting of a list of 142 ''curricular'' Milestones (which are distinct from the 22 internal medicine reporting Milestones).9,10 These curricular Milestones represent specific, observable skills in the 6 competencies residents should demonstrate at each level of training during the 3 years of training. Although program directors have developed new assessment tools based on the Milestones, little is known about how best to educate and support faculty in using the new tools. To date, research has not examined the effect of faculty development in the use of a Milestone-based evaluation form. We sought to determine whether faculty development would reduce common rating errors among faculty completing Milestone-based end-of-rotation evaluations. Our hypothesis was that faculty who participated in faculty development AbstractBackground Rater errors, such as halo/reverse halo, range restriction, and leniency errors, are frequently cited as threats to the validity of resident assessment by faculty.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.