Student Evaluation of Teaching (Set) Ratings Depend on the Class Size: A Systematic Review

Abstract

Feldman (1984) reviewed studies investigating the relationship between SET ratings and the class size and concluded that their association was "very weak" and inverse. However, a detailed review of Feldman's meta-analysis reveals a number of serious methodological flaws that undermine his conclusion. Briefly, Feldman excluded studies that reported curvilinear relationships and did not consider a possibility that SET ratings may be the highest in smallest classes, decline sharply as the class size increases to 20 or 30 students and then level off; and that correlations reported in primary studies may be artifacts of calculating linear correlation coefficients over non-linear relationships. Accordingly, we conduct a new, up-to-date, comprehensive meta-analysis of the SET/Class size relationships using data from over 100 primary studies. The key results of our research show that (1) the primary studies frequently did not report necessary data to interpret their findings, and (2) the SET/Class size relationship is curvilinear, specifically, SET ratings are the highest in smallest classes, decline sharply as the class size increases to 20 to 30 students, and then level off revealing minimal SET/Class size relationship among studies with class sizes beyond 50 students.

Spain is not different: teaching quantitative courses can also be hazardous to one’s career (at least in undergraduate courses)

“…Discrepancies are considerable, although we are analyzing the same group of instructors in both cases. Certainly, given that class size in master courses is significantly smaller, it is possible that part of the difference is due to this bias (see Uttl, Bell & Banks, 2018 ). However, the differences are too large for that to be the explanatory factor, especially when areas such as “management”, “general contents,” and “law” do not see the same effect.…”

“…Additionally, Arroyo-Barrigüete et al (2021) , using data from the Universidad Pontificia Comillas, concluded that instructors offering easy courses (low workload) tend to be rated highly. Uttl, Bell & Banks (2018) showed that class size also affects SETs, with a curvilinear relationship; i.e ., SET ratings are the highest in the smallest classes, decline as the class size increases to 20–30 students, and then level off.…”

Lessons Learned from Research on Student Evaluation of Teaching in Higher Education

“…Our review (Uttl et al, 2018) of over 100 studies that examined the relationship between SET and class size, including those reviewed by Feldman (1984), revealed that the vast majority of these studies did not report sufficient information to interpret their findings. For example, many studies did not report the smallest class size, did not report the largest class size, did not report the number of classes within each class size category, did not examine the linearity of SET/class size relationship, did not examine whether there was a decline in SET for classes with fewer than 20 or 30 students, did not show scatterplots of SET/class size relationships, had very small sample sizes, included extreme outliers, etc.…”