Online student evaluations of teaching: what are we sacrificing for the affordances of technology?

Abstract

In the context of increased emphasis on quality assurance of teaching, it is crucial that student evaluation of teaching (SET) methods are both reliable and workable in practice. Especially, online SET tends to raise criticisms with those most reactive to mechanisms of teaching accountability. However, most studies have been conducted with convenience, small and cross-sectional samples. Longitudinal studies are rare, as comparison studies on SET methodological approaches are generally pilot studies followed shortly after by implementation. The investigation presented here significantly contributes to the debate by examining the impact of the online administration method of SET on a very large longitudinal sample at course level. The study explores the impact of the administration method of student evaluations of teaching (paper based in-class versus off-class online collection) on scores with a longitudinal sample of over 63,000 student responses collected over a total period of ten years. Having adjusted for the confounding effect of class size, faculty, year of evaluation, years of teaching experience and student performance, it is observed that the actual effect of the administration method exists, but is insignificant.

Exploring ethical issues associated with using online surveys in educational research

“…Reviews conducted to date suggest that equivalent results are obtained using online and offline surveys (Gosling et al, 2004;Roberts, 2006), with the possible exception of measures that may be subject to social desirability response sets (Roberts, 2006). For example, based on a 10 year longitudinal study of 63,000 student responses to SETs, Risquez, Vaughan, and Murphy (2014) reported that after controlling for class size, faculty, year of evaluation, years of teaching experience and student performance, the effect of administration mode on SET results is minimal. Further, reviewing metaanalyses of non-response bias studies, Peytchev (2013) noted that there is little evidence of a relationship between response rate and non-response bias.…”

La Técnica Grilla de Repertorio (Repertory Grid): Un Método para el Estudio de la Cognición en la Evaluación de Habilidades Docentes

“…Tercero, este trabajo potencia la evaluación docente basada en cuestionarios, dado que a partir de los constructos elicitados será factible especificar instrumentos de evaluación docente, pero reconociendo que los estudiantes usan sus propios constructos para entender e interpretar eventos que suceden a su alrededor y que estos constructos están moderados por sus experiencias individuales. Cuarto, si bien RepGrid fortalece el método de los cuestionarios en el proceso de la evaluación docente, su efectividad final está determinada por su utilización en conjunto con otros métodos de evaluación del desempeño (Risquez, Vaughan, y Murphy, 2015). Por ejemplo, evaluación de pares y portafolios de evidencias (Alderman, Towers, y Bannah, 2012).…”

Normalisation of Technology Use in a Developing Country Higher Education Institution

“…The cost of printing, administering and processing the survey results were the most compelling reasons. The time savings in terms of administration and processing of feedback was also key [1]. The time savings related to students too because they would have some time in their own time to reflect on their answers before they submit [9].…”

“…Despite the limitations of high financial cost, waste of time and problems with analysis the paper-based evaluations are widespread [4], [5] in developing countries. As a result, many HEIs in developing countries are migrating to online evaluations where students use online forms [1], [4]. A lot of studies on the course and lecturer evaluation in HEIs [6]- [8] have largely been quantitative with less qualitative studies.…”

“…Course and lecturer evaluation is the most commonly used method of assessing a course and lecturer effectiveness because it offers important opportunities for feedback and development [1], and has been routinely used in academic institutions to inform curricular change and assess lecturer's performance [2]. Whilst course and lecturer evaluations may be paper-based or virtual, it has largely been conducted in class at the end of the academic semester through the use of paper evaluation forms [3] in many developing countries.…”