Improving response rates for course and instructor evaluations using a global approach

Abstract

Policy Pearls intRoductionStudents perspectives on their instructors and courses (student ratings of instruction, student ratings of teaching, student evaluation of teaching (here referred to as SETs)) continue to be a topic of extensive research and debate. [1][2][3][4] The use of SETs is expanding globally with increasing value placed on the data for promotion, tenure, and retention. [5] While SETs are not infallible and should not be used in isolation, most agree that student input is essential for evaluation of teaching and course effectiveness. [6][7][8] Many suggest that the problems associated with these surveys have small effects on the final score as long as enough information is collected and interpretation is appropriate. [7][8][9] Adequate response rates for the population are a major key to valid results, minimizing the risk of sampling errors and biased results. [8,10,11] SETs are required at the University of Minnesota by university policy as well as by departmental and collegiate promotion and tenure guidelines. Each semester, students are asked to rate each of their courses and instructors. For many years, course evaluation forms included nine separate criteria and instructor evaluations included twelve. Comments were requested for each of the categories. For our class sizes (approximately 100 students per cohort) and survey design, a >50% response rate was calculated as necessary for reasonable score validity when the average score variability (SD) was ≤1.0. [10,11] For many years, we experienced such low response rates that score validity was in question. Multiple methods were attempted to improve response rates including using forms with fewer questions, only requesting surveys if an instructor taught more than three sessions in a course, opening surveys mid-semester for earlier input, offering rewards for high participation rates, prize drawings, and attempts to withhold grades until surveys were returned. Despite these efforts, response rates continued to decline. However, without an obvious alternative, the data were still used by the curriculum committee for course decision-making and were considered for promotion and salary decisions by departments.In recent years, we changed our practices and significantly improved both course and instructor evaluation response rates and data quality in a time-efficient manner. The purpose of this article is to describe this model to help educators at other Obtaining sufficient survey responses to make course and instructor evaluation results meaningful is a challenge in many, if not most, health professions training programs. This paper describes a series of policy changes that significantly improved data quality at one college of veterinary medicine located in the United States. The steps consisted of minimizing the number of items appearing on the instruments, providing students adequate time and space for completion, clearly explaining the purpose and value of the evaluations, simplifying data collection, collecting verbal feedback, and closing the l...

Student evaluations of teaching and the development of a comprehensive measure of teaching effectiveness for medical schools

“…Therefore, it may not always be clear what the questions are asking or what is meant by the students' comments or scores provided in SETs [ 48 ]. Several institutions have addressed this matter by gathering students for an hour and providing designated time to complete SETs online (anonymously) [ 49 ]. In their study Malone et al (2018) [ 49 ] conducted sessions in which they discussed with the student the purpose of SETs, who will see the results of SETs and how the results will be used, as well as any other matters raised by the students.…”

“…Several institutions have addressed this matter by gathering students for an hour and providing designated time to complete SETs online (anonymously) [ 49 ]. In their study Malone et al (2018) [ 49 ] conducted sessions in which they discussed with the student the purpose of SETs, who will see the results of SETs and how the results will be used, as well as any other matters raised by the students. SETs were treated much like a focus group in which open-ended questions were answered by the collective group through informal polling, debate, and clarification.…”

“…SETs were treated much like a focus group in which open-ended questions were answered by the collective group through informal polling, debate, and clarification. Response rates exceeded 70–80% thus providing faculty, administrators and curriculum committees substantial information [ 49 ].…”

“…As previously discussed, this is a common problem in medical school courses where each course may be delivered by many faculty members; yet students may not remember who contributed to each course and the quality of teaching exhibited by each faculty member. This problem has been addressed by some institutions by providing a photo of each instructor, his/her name and topics/ dates presented to refresh students’ memories and to avoid issues compromising the validity of the SETs [ 49 ].…”

The Kubler-Ross change curve and the flipped classroom: Moving students past the pit of despair

“…Some students become stuck at this stage, and we only learn of their frustration when we review comments from end-of-course evaluations. [23] Further complicating matters, students know most courses begin by covering fundamental concepts and grow progressively more challenging over time. As students become increasingly familiar with the flipped classroom format and develop the skills necessary for success in this format, the learning experience becomes less stressful.…”