Multilevel Analysis of the Effects of Student and Instructor/Course Characteristics on Student Ratings

Abstract

Multilevel SEM was used to examine the extent to which student, instructor, and course characteristics affect student ratings. Data were gathered from 1867 students enrolled in 117 courses at a large teacher training college in Israel. Four alternative two-level models that differ in only the nature of the relationship among interest in the course subject, expected grade, and student ratings were tested. Two of the models were judged as less appropriate, one because it failed to support the spurious relationship assumed between expected grade and student ratings, and the other on grounds of poor model-data fit. The other two models were equally good both in terms of the model-data fit and the amount of variance in student ratings that is accounted for by each of them. Both models supported the mediation effect of expected grade in the relationship between interest in the course subject and student ratings.

Taking the grading leniency story to the edge. The influence of student, teacher, and course characteristics on student evaluations of teaching in higher education

“…For this reason, we add the group-mean centered characteristics in a separate model. Although many previous studies have used uni-level regression models to estimate the influence of actual or expected grades on SET scores, there is a tendency for SET data to be regarded as a multilevel phenomenon (Marsh and Overall 1981;Nasser and Hagtvet 2006). Estimating a multilevel model is beneficial, in that it takes the hierarchical structure of the data into account.…”

Multilevel analysis of the effects of student and course characteristics on satisfaction in undergraduate liberal arts courses

“…Most of the research on this issue agrees that the higher grades student expect, the higher ratings of instruction appear. However, this relationship can be interpreted as either student's preference for faculty or student's active course participation (Baek et al, 2005;Marsh & Roche, 2000;Nasser & Hagtvet, 2006).…”

“…On the other hand, if the mean score of each class were used, an aggregation bias appears. In order to solve these problems, a hierarchical linear model (HLM), which analyzes not only student-level data but also course-level data, might be more appropriate (Chin, 2007;Civian & Brennan, 1996;Nasser & Hagtvet, 2006;Raudenbush & Bryk, 2002;Tabachnick & Fidell, 2007;Ting, 2000;Umbach & Porter, 2002). The HLM is a data analysis technique for research designs where the data for participants is organized at more than one level.…”

The Tail Wagging the Dog; An Overdue Examination of Student Teaching Evaluations

“…Yet, in a later meta-analysis of more than 41 studies, independently examining the relationship between course grades and STEs, found that indeed expected course grades were significantly and positively correlated with teaching effectiveness rating; providing some evidence for the influence of course grades and STEs (Cohen, 1981). Since this study, empirical research incorporating almost 40 years suggests STEs are higher when a student expects to earn a higher grade (Feldman, 1976, Goldberg & Callahan, 1991Marsh & Rochi, 1997;Nasser & Hagtvet 2006). Yet, some controversy exists around the roll of student engagement.…”