Charting new territory: assessing the online frontier of student ratings of instruction

Abstract

Computer-mediated communication pervades society today. Higher education is no exception. Recently in the USA, a debate has emerged regarding online administration of student ratings of instruction (SROIs). Since these end-ofsemester messages offered by students are used to influence merit, promotion and tenure decisions, the utility of administering them online ought to be examined. This comparative analysis of messages communicated by students via online and paper-and-pencil SROIs revealed several conclusions. First, the means by which online data were collected positively influenced overall student response rate, as well as internal question-by-question response rate. Second, no significant differences were revealed in (1) student responses to the Likert-type scale assessments, (2) number of comments provided to the open-ended questions, (3) number of positively and negatively charged comments offered in the open-ended responses or (4) major content themes addressed in the open-ended comments.Finally, students provided more descriptive detail in the online than the paper-andpencil responses. When administered effectively, collecting student rating messages online may be better than doing so in the traditional paper-and-pencil format.

A Systematic Review of Online Response Rates in Four Counseling Journals

“…For example, faulty email addresses can significantly alter sample size, offsetting the cost savings often attributed to web-based surveys over mail-based surveys (Jacob, 2011). Lower response rates in web-based surveys still could potentially reflect a participant's lack of familiarity with technology, lack of internet access, or lack of knowledge (Venette, Sellnow, & McIntyre, 2010).…”

An Empirical Look at the Impact of Course and Faculty Characteristics on Student Evaluations

“…While there are some advantages to using online student evaluations, such as more thorough comments, one disadvantage is its lower response rate compared to traditional paper-and-pencil evaluations (Spooren, et al, 2013). Venette, Sellnow, and McIntyre (2010) offer an example to the contrary, where online evaluations received higher response rates, and confirmed that open-ended comments were more thorough for online evaluations. Meanwhile, other scholars found no differences in the number of comments between online and onsite evaluations (Stowell, et al, 2012).…”

“…The literature talks at length about the way students complete evaluations, online or on-site via paper and pencil, and the pros and cons of each methodology (Hoffman, 2003;Spooren, Brockx, & Mortelmans, 2013;Stowell, Addison, & Smith, 2012;Venette, Sellnow, & McIntyre, 2010). However, there are fewer discussions about the impact of instructional method on student evaluations.…”

“…For one, the university is relying on individuals to complete these evaluations outside of class and usually after the course is completed. Institutions do not typically require participation and there has been a noticeable decline in online response rates compared to the traditional paper and pencil evaluations (Stowell, Addison, & Smith, 2012;Spooren, Brockx, & Mortelmans, 2013), though some scholars have found online evaluations to yield a higher response rate (Venette, Sellnow, & McIntyre, 2010).…”

“…In terms of open-ended questions on these evaluations, students who complete the online evaluations tend to give more thorough and thoughtful responses (Venette, et al, 2010), which are valued by many as more insightful than the responses to the Likert-scale questions on the evaluations. Additionally, online evaluation tools make it easier to organize collected data and examine changes in student responses for a particular faculty member or course over time, since most analysis is already completed via computers.…”