Factors affecting the student evaluation of teaching scores: evidence from panel data estimation

Abstract

We use a random-effects model to find the factors that affect the student evaluation of teaching (SET) scores. Dataset covers 6 semesters, 496 undergraduate courses related to 101 instructors and 89 disciplines. Our empirical findings are: (i) the class size affects negatively the SET score; (ii) instructors with more experience are better evaluated, but these gains reduce over time; (iii) participating in training programs, designed to improve the quality of teaching, did not increase the SET scores; (iv) instructors seem to be able to marginally 'buy' a better evaluation by inflating students' grade. Finally, there are significant changes in the rankings when we adjust the SET score to eliminate the effects of variables beyond instructors' control. Despite these changes, they are not statistically significant. Keywordsstudent evaluation score, random-effects model, undergraduate, ranking JEL Classification A22 ResumoEste trabalho emprega um modelo de efeitos aleatórios para estimar os principais fatores determinantes na avaliação de professores por estudantes. Os dados compreendem ♦ We are grateful to Marcia Moura for authorizing the use of the data for this study. We would like to thank Carolina Costa, Tadeu Ponte and specially Rogério Costa for making available the data used for this study. Any views expressed are those of the authors' exclusively. The views expressed herein are those of the author(s) and do not necessarily reflect the views of the Insper. We also would like to thank the three anonymous referees for helpful comments. Needless to say, remaining errors and omissions are of our responsibility.Est. Econ., São Paulo, vol. 42, n.1, p. 129-150, jan.-mar. 2012 130 Eduardo de Carvalho Andrade e Bruno de Paula Rocha496 cursos na graduação, relacionados a 101 diferentes professores e 89 disciplinas, durante 6 semestres letivos. Os principais resultados obtidos são: (i) o tamanho das salas de aula afeta negativamente a nota recebida pelo professor; (ii) professores mais experientes são mais bem avaliados, mas estes ganhos são decrescentes ao longo do tempo; (iii) a participação em programas de treinamento, desenhados para melhorar a qualidade do ensino, não afetam as notas recebidas pelos professores; (iv) a avaliação recebida pelos professores é marginalmente influenciada pela nota dada aos alunos durante o curso. Finalmente, os dados mostram mudanças significativas no ranking dos professores, quando são feitos ajustes para eliminar os efeitos de variáveis fora do controle dos professores. Entretanto, estas mudanças não são estatisticamente significativas.

Research enchancing the quality of teaching: Econometric evidence from Brazil

“…7 As can be seen, a higher average grade (GRADE) in the course positively affects the SET. The same result is obtained in Andrade & Rocha (2012), indicating that a professor may "buy" a better evaluation by inflating the student's grade. Teaching experience (EXP) also positively influences the SET.…”

“…Evidence from Brazilian data can be found in Andrade & Rocha (2012). sign of this coefficient probably reflects an "award" due to the higher level of attention given by the professor to any particular student in courses with small classes (Andrade & Rocha 2012). In model (4), the coefficient of the variable AGE seems to suggest that, controlling for experience (EXP), there is no evidence of relevant human capital depreciation or student's discrimination against older professors.…”

“…7 For a discussion of the effects of these variables on SET, see McPherson et al (2009). Evidence from Brazilian data can be found in Andrade & Rocha (2012). sign of this coefficient probably reflects an "award" due to the higher level of attention given by the professor to any particular student in courses with small classes (Andrade & Rocha 2012).…”

“…Teaching experience (EXP) also positively influences the SET. This positive effect may reflect the professor's adaptation to the institution's environment and the student behavior (Andrade & Rocha 2012). On the other hand, a greater class size (CSIZE) negatively affects the SET.…”

“…The 5 The list of questions that compose the SET form is: "organization and clarity", "communication skills", "exams format", "course content", "academic rigor", "academic experience", "market experience", "pragmatism", "interaction", "stimulating attitude", "stimulus for studies", "classroom dynamics" and "dedication". A more detailed description of the SET forms can be seen in Andrade & Rocha (2012).…”