RateMyProfessors.com offers biased evaluations

Abstract

RateMyProfessors.com (RMP) is becoming an increasingly popular tool among students, faculty and school administrators. The validity of RMP is a point of debate; many would argue that self-selection bias obscures the usefulness of RMP evaluations. In order to test this possibility, we collected three types of evaluations: RMP evaluations that existed at the beginning of our study, traditional in-class evaluations and RMP evaluations that were prompted after we collected in-class evaluations. We found differences in the types of evaluations students provide for their professors for both perceptions of professor clarity and ratings of professor easiness. Based on these results, conclusions drawn from RMP are suspect and indeed may offer a biased view of professors. IntroductionStudent evaluations are an important part of the feedback process for university professors. Implications of positive student evaluations include pay raises for professors, advancement to tenure status and improved marketability when applying for new teaching positions. Student evaluations also offer professors vital feedback to enhance teaching. For these reasons, the importance of student evaluations is evident.Universities and colleges have several options regarding how to administer their student evaluations of professors. It is common for universities to use traditional inclass evaluations in which students use pencil and paper to provide feedback about their professors. However, as more universities begin to move towards a paperless environment, institutions may instead use online evaluations. Most online evaluations offer students a chance to provide feedback about professors at a location and time of their choice, reduce time constraints and reduce potential influence from the professor (Anderson, Cain, and Bird 2005). Online evaluations also decrease the problem of students who do not get the chance to complete in-class evaluations if they miss class during evaluation day.The use of online evaluations versus traditional in-class evaluations raises questions regarding similarity in responses across the two methods of presentation. Student ratings of professors have been shown to be similar across the two formats when the professor requests the evaluations (Donovan, Mader, and Shinsky 2006). However, Donovan and colleagues found that the content of comments provided by students differed depending upon whether the evaluation was online or in class.

The Online Evaluation of Courses: Impact on Participation Rates and Evaluation Scores

“…However, research on this subject shows that this is not the case, and that these ideas represent misconceptions about the online evaluation of courses (University of Saskatchewan, n.d.). In fact, according to several studies, the judgements made by students about courses through online evaluations appear to be similar to those made using paper-based evaluations, and this despite a lower participation rate (Gamliel & Davidovitz, 2005;Legg & Wilson, 2012;Nowell et al, 2010;Stowell et al, 2012; University of Saskatchewan, n.d.; Venette, Sellnow, & McIntyre, 2010).…”

Students' perspectives on RateMyProfessors.com: an empirical investigation of perception and attitude

“…Some have found RMP ratings to be biased (Clayson, 2014;Legg and Wilson, 2012), whereas others have found them to be valid measures of student learning (Otto et al, 2008). A common concern of reliability and validity is that the characteristics of those self-elect individuals who provide RMP evaluations could be different from all students (Legg and Wilson, 2012) and evaluations may be influenced by other factors such as race, gender, attractiveness and easiness of the instructor than quality of teaching (Felton et al, 2004(Felton et al, , 2008Leung et al, 2013;Reid, 2010;Stuber et al, 2009). Students motivated to post their evaluations could possibly have extreme viewpoints about the professor (Peterson et al, 2011).…”

Affordances and negotiations of the digital reputation society: a case study of RateMyProfessors.com

“…Studying teaching review sites: participatory or commodified agency While most studies of popular teaching evaluation sites focus on their validity, reliability and bias (Kindred and Mohammed 2006;Coladarci and Kornfield 2007;Helterbran 2008;Davison and Price 2009;Sonntag, Bassett, and Snyder 2009;Reid 2010;Legg and Wilson 2012), few recent studies have explored, even partially, the sociocultural meanings of RMP as a popular phenomenon (e.g. Reagan 2009;Ritter 2008;Chaney 2011;Gregory 2012).…”