Forecasting the student–professor matches that result in unusually effective teaching

Abstract

BackgroundTwo important influences on students' evaluations of teaching are relationship and professor effects. Relationship effects reflect unique matches between students and professors such that some professors are unusually effective for some students, but not for others. Professor effects reflect inter-rater agreement that some professors are more effective than others, on average across students.AimsWe attempted to forecast students' evaluations of live lectures from brief, video-recorded teaching trailers.SampleParticipants were 145 college students (74% female) enrolled in introductory psychology courses at a public university in the Great Lakes region of the United States.MethodsStudents viewed trailers early in the semester and attended live lectures months later. Because subgroups of students viewed the same professors, statistical analyses could isolate professor and relationship effects.ResultsEvaluations were influenced strongly by relationship and professor effects, and students' evaluations of live lectures could be forecasted from students' evaluations of teaching trailers. That is, we could forecast the individual students who would respond unusually well to a specific professor (relationship effects). We could also forecast which professors elicited better evaluations in live lectures, on average across students (professor effects). Professors who elicited unusually good evaluations in some students also elicited better memory for lectures in those students.ConclusionsIt appears possible to forecast relationship and professor effects on teaching evaluations by presenting brief teaching trailers to students. Thus, it might be possible to develop online recommender systems to help match students and professors so that unusually effective teaching emerges.

Understanding the P×S Aspect of Within-Person Variation: A Variance Partitioning Approach

“…Other investigators have documented P×S effects for memory performance following training ( Gross et al, 2009 , 2015 ). Persons heard presentations from different trainers (stimuli) and were tested on retention.…”

“…There are large P×S effects for students’ (persons) evaluations of instructors’ (situations) teaching ( Gross et al, 2009 , 2015 ). That is, Student A might find Instructor A to be more effective than Instructor B, but Student B might have the opposite opinion.…”

“… Gross et al (2015) tested this concept by developing brief videos of instructors’ teaching. The teaching trailers (cf.…”

“…Forecasting the P×S aspect of teaching effectiveness involved mapping each student’s profile of responses to the trailers at Time 1 to his profile of responses to lectures at Time 2. In fact, Gross et al (2015) could accurately forecast the instructors that specific students found unusually effective.…”

“…That is, a person’s reaction to a stranger from a brief conversation forecasted the extent to which the person ultimately saw the former stranger as unusually supportive weeks and months later. Veenstra et al’s (2011) analytic approach was the same as in Gross et al (2015) . From the first conversation (Time 1), each person had a profile of scores across the providers.…”