Registered Replication Report

Abstract

Trying to remember something now typically improves your ability to remember it later. However, after watching a video of a simulated bank robbery, participants who verbally described the robber were 25% worse at identifying the robber in a lineup than were participants who instead listed U.S. states and capitals-this has been termed the "verbal overshadowing" effect (Schooler & Engstler-Schooler, 1990). More recent studies suggested that this effect might be substantially smaller than first reported. Given uncertainty about the effect size, the influence of this finding in the memory literature, and its practical importance for police procedures, we conducted two collections of preregistered direct replications (RRR1 and RRR2) that differed only in the order of the description task and a filler task. In RRR1, when the description task immediately followed the robbery, participants who provided a description were 4% less likely to select the robber than were those in the control condition. In RRR2, when the description was delayed by 20 min, they were 16% less likely to select the robber. These findings reveal a robust verbal overshadowing effect that is strongly influenced by the relative timing of the tasks. The discussion considers further implications of these replications for our understanding of verbal overshadowing.

Participant Nonnaiveté and the reproducibility of cognitive psychology

“…In addition, there have been several been high-profile, preregistered, multi-lab failures to replicate wellknown effects psychology (Eerland et al, 2016;Hagger et al, 2016;Wagenmakers et al, 2016). A similar multi-lab replication psychology that was considered successful yielded an effect size that was much smaller than the original (Alogna et al 2014). These findings have engendered pessimism about reproducibility.…”

A New Replication Norm for Psychology

“…The more findings you publish, the more independent replications you need to execute and publish for everyone to benefit from a self-correcting cumulative knowledge base. 2 Another parallel between the peer review and replication norm worth mentioning is that in each case, researchers' primary motivation to embrace the norm is that they intrinsically care about the theoretical progress of their own research area. Hence, even though engaging in such activities takes time away from doing their own research, it is nonetheless in their best interest to peer-review and replicate other researchers' findings.…”

“…For instance, Perspectives on Psychological Science (PPS) now offers a new article type called Registered Replication Reports that involve multi-lab pre-registered replication attempts of important findings in psychology [90]. The process involves one lab submitting a proposal to replicate a finding that is deemed important (either theoretically influential or having important societal implications) which has yet to be independently replicated and/or still has substantial uncertainty about the size of the effect (e.g., multisite replication of verbal overshadowing effect [2]). Once a replication proposal is accepted and all procedures/ materials are finalized, PPS makes a public announcement, at which time other labs (typically about 15) can join in to contribute a sample to the replication effort and earn co-authorship on the final article.…”

“…However, given that revised versions of rejected manuscripts submitted to a different journal are often reviewed by some of the same reviewers (and involve less work to review subsequently), for the sake of simplicity, it would appear researchers calibrate their peer review behavior in reference to number of published rather than submitted articles. 2 It is important to mention that the difference between original and replication research should not be exaggerated given that in actuality a continuum exists between original and replication research [1]. 3 Also, like the peer review norm, researchers may naturally calibrate their required contribution to the system depending on their faculty position and available resources (e.g., research vs. teaching position, access to large vs. small subject pools, etc.).…”

Collections2: Using “Crowdsourcing” within Psychological Research

“…For example, several recent, high-profile, large-scale research projects have demonstrated the potential of crowdsourcing research resources to make substantial contributions (e.g., the "Many Labs" projects, Ebersole et al, 2016;Eerland et al, 2016;Klein et al, 2014;Registered Replication Reports [RRR], Alogna et al 2014;Cheung et al, 2016;Hagger et al, 2016;Wagenmakers et al, 2016;"The Pipeline Project," Schweinsberg et al, 2016; the "ManyBabies" project, Frank et al, 2017; see also Schmalz, 2016). In each of these projects, several research teams each conducted a study (a) following the same methods, (b) at different locations and with different samples, and (c) the results from each research team were aggregated into a planned, common analysis from the project's inception.…”