A comparison of online versus traditional student end‐of‐course critiques in resident courses

Abstract

This study outlines the intraclass differences between online and traditional student end-of-course critiques (EOCCs) only in the context of resident courses. Previous research into these differences appears limited to studies that compare entire classes rather than studies within any given class. Hypotheses are stated that the administration method has an effect on: how students will rate EOCCs, EOCC response rates, and, the detail level, favourability level and number of EOCC comments. In this field experiment, students in resident business courses at a large university comprised the sample. Individuals from within each class were randomly assigned either to a control group that completed a traditional paper-based EOCC or to an experimental group that completed a similar EOCC online. Comments were coded with regard to the level of detail, favourability and total number of comments. Analysis of variance was used to compare the two groups with regard to ratings, comments and response rates. Online EOCCs had lower response rates, lower overall ratings, but more detailed comments. The methodology had no significant effect on the number or favourability level of comments. Literature review and research hypothesesThe university used as the field setting for this study is a state-funded university located in the Southeastern USA, with numerous satellite campuses located in nearly a dozen countries and as many states. Total enrolment exceeds 27,000 graduate and undergraduate students. In the fall semester of 2000, this university began a state-mandated change from a quarter to a semester system. For a large university, making paradigm shifts such as these presents numerous challenges. One such challenge that remains unanswered has been how to conduct student end-of-course critiques (EOCCs) in a manner that produces usable and reliable data in a global university. Although traditional methods of gathering student evaluations of learning environments proved sufficient in the past, the global nature of the new system requires commonality among the different satellite campuses. For example, separate promotion and tenure committees were found on several campuses, but in the new system, a single promotion and tenure committee is located on the main campus. Committee members travel to the main campus for committee meetings or they participate via videoconferencing, teleconferencing, or even by email for specific agenda items. To prevent confusion and to ensure fairness, standardised methods are necessary throughout the university *

Beneficios de la distribución de cuestionarios web de valoración docente a través de mensajería SMS en el ámbito universitario: tasas de participación, inversión de tiempo al completar el cuestionario y plazos de recogida de datos

“…Este problema se ha visto acentuado con la introducción de cuestionarios web. Diversas investigaciones revelan que las tasas de participación alcanzadas en valoraciones administradas con instrumentos online son inferiores a aquellas desarrolladas en papel (Morrison, 2011;Nowell, Gale, & Handley, 2010;Nulty, 2008;Stanny & Arruda, 2017;Stowell, Addison, & Smith, 2012). Tanto es así, que esta caída en las ratios de respuesta ha sido durante años una de las objeciones frecuentemente esgrimidas en el ámbito universitario para no abordar la transición de las evaluaciones tradicionales a las mediciones online (Crews & Curtis, 2011).…”

Online student evaluations of teaching: what are we sacrificing for the affordances of technology?

“…The general consensus is that, while the electronic administration method almost invariably results in lower response rates, its impact on results is non existent or minimal (Ardalan et al 2007;Avery et al 2006;Dommeyer et al 2004;Dresel and Tinsner 2008;Johnston 2003;Layne et al 1999;Liegle and McDonald 2005;Perrett 2011;Stowell et al 2011;Winer and Sehgal 2006). However, some recent conflicting evidence has also been presented (Morrison 2011;Nowell et al 2010) that reports lower average SET ratings from evaluations conducted online. As a result, these authors argue that moving to online evaluation is not overly problematic, yet it is not comparable to in-class administration and as a result, institutions should choose one method or the other.…”

“…But by far, the main reason why the progressive transition towards online administration methods in the last decade has challenged the credibility of SET relates to sample representation. It is widely acknowledged that, provided there is high class attendance -which in itself is a substantial assumption-, traditional paper-and-pencil administrations tend to yield higher response rates (Avery et al 2006;Bennett and Nair 2010;Dresel and Tinsner 2008;Morrison 2011;Nulty 2008;Perrett 2011;Sax et al 2003). This is due, of course, to the use of a 'captive audience' targeted during in-class evaluations, where the instructor or personnel from an educational development centre circulate and collect evaluation forms.…”

Student Perceptions of Online Learning: An Analysis of Online Course Evaluations

“…Early research found that teaching evaluations completed online did not differ from evaluations completed in a traditional format (Avery et al 2006;Donovan, Mader, and Shinsky 2010;Stowell, Addison, and Smith 2012). In fact, some research suggests that students leave even more feedback (Donovan, Mader, and Shinsky 2010;Hardy 2003;Hmieleski and Champagne 2000;Kasiar, Schroeder, and Holstaad 2002;Morrison 2011;Stowell, Addison, and Smith 2012) and at times even more positive feedback (Carini et al 2003) when completing evaluations online. Some research even revealed that students prefer completing online evaluations (Hardy 2003).…”