Psychology researchers have long attempted to identify educational practices that improve student learning. However, experimental research on these practices is often conducted in laboratory contexts or in a single course, which threatens the external validity of the results. In this article, we establish an experimental paradigm for evaluating the benefits of recommended practices across a variety of authentic educational contexts—a model we call ManyClasses. The core feature is that researchers examine the same research question and measure the same experimental effect across many classes spanning a range of topics, institutions, teacher implementations, and student populations. We report the first ManyClasses study, in which we examined how the timing of feedback on class assignments, either immediate or delayed by a few days, affected subsequent performance on class assessments. Across 38 classes, the overall estimate for the effect of feedback timing was 0.002 (95% highest density interval = [−0.05, 0.05]), which indicates that there was no effect of immediate feedback compared with delayed feedback on student learning that generalizes across classes. Furthermore, there were no credibly nonzero effects for 40 preregistered moderators related to class-level and student-level characteristics. Yet our results provide hints that in certain kinds of classes, which were undersampled in the current study, there may be modest advantages for delayed feedback. More broadly, these findings provide insights regarding the feasibility of conducting within-class randomized experiments across a range of naturally occurring learning environments.

No abstract

This study examined the extent to which predicted responses to ANSI Z535-style warning signs and labels correspond to actual responses. In previous studies, lay people's impressions of the influence of warning attributes have not been found to accurately predict actual responses to warnings or to reflect the effects obtained in behavioral studies for different warning conditions. However, these studies have not specifically focused on the connection between predicted and actual responses to warnings that vary according to conformance with an ANSI standard for product warnings. Four warning scenarios were employed in the present study, with warnings varying according to the extent of ANSI-conformance. Participants consistently and incorrectly predicted that people would be more likely to notice and/or comply with warnings that were more consistent with ANSI as opposed to those warnings that were less-consistent with ANSI. Collectively, people's perception of differences in response to warnings was not consistent with what has been reported in observational studies.

Psychology researchers have long attempted to identify educational practices that improve student learning. However, experimental research on these practices is often conducted in laboratory contexts or in a single class, threatening the external validity of the results. In this paper, we establish an experimental paradigm for evaluating the benefits of recommended practices across a variety of authentic educational contexts – a model we call ManyClasses. The core feature is that researchers examine the same research question and measure the same experimental effect across many classes spanning a range of topics, institutions, teacher implementations, and student populations. We report the first ManyClasses study, which examined how the timing of feedback on class assignments, either immediate or delayed by a few days, affected subsequent performance on class assessments. Across XX classes, [summarize effect of feedback timing, including key moderators]. More broadly, these findings provide evidence regarding the feasibility of conducting within-class randomized experiments across a range of naturally occurring learning environments.

No abstract