Validation of a research-based student survey of instruction in a college of education

Abstract

As a preparer of students for the profession of teaching, a college, school, or department of education in higher education has an expectation that instructors teach competently. In higher education, the student survey of instruction is one of the most important among other indicators of teaching effectiveness. This article presents a validation of a new student survey of instruction in a college of education. In this validation, we found the new survey to be as troublesome as the previous survey. Four major findings emerged from this study that offer hope for a new and improved survey of instructional effectiveness. These findings involve estimating reliability, subscore validity, course relevance and rigor of the instructor, and construct-irrelevant variables that threaten validity. Given the failed validation, many cogent observations provide guidance for the design and development of improved forms that may overcome these threats to validity and guide others in validating teaching effectiveness surveys.

On the Validity of Student Evaluation of Teaching

“…Several well-designed and validated instruments are available, however, including the Instructional Development and Effectiveness Assessment (IDEA; Cashin & Perrin, 1978), the Students' Evaluation of Education Quality (SEEQ; Marsh, 1982;Marsh et al, 2009), the Course Experience Questionnaire (CEQ; Ramsden, 1991), the Student Instructional Report (SIR II; Centra, 1998), and the Student Perceptions of Teaching Effectiveness (SPTE; Burdsal & Bardo, 1986;Jackson et al, 1999), as well as the more recent Students' Evaluation of Teaching Effectiveness Rating Scale (SETERS; Toland & De Ayala, 2005), the Student Course Experience Questionnaire (SCEQ; Ginns, Prosser, & Barrie, 2007), the Teaching Proficiency Item Pool (Barnes et al, 2008), the SET37 questionnaire for student evaluation of teaching (SET 37, Mortelmans & Spooren, 2009), the Exemplary Teacher Course Questionnaire (ECTQ; Kember & Leung, 2008), and the Teaching Behavior Checklist (Keeley, Furr, & Buskist, 2010;Keeley, Smith, & Buskist, 2006). Validation procedures for other instruments have not been successful (Haladyna & Amrein-Beardsley, 2009). Still, many instruments are developed without any clear theory of effective teaching (Ory & Ryan, 2001;Penny, 2003).…”

Taking the grading leniency story to the edge. The influence of student, teacher, and course characteristics on student evaluations of teaching in higher education

“…For example, Theall and Franklin (2001) state that "few issues in higher education are as sensitive, divisive and political as faculty evaluation and in particular the quality and value of the information provided by students in their evaluations of teachers and courses" (p. 45). Nonetheless, SET has become common practice in most institutions of higher education, and it is considered a highly valuable method of evaluating teaching (Haladyna and Amrein-Beardsley 2009).…”