Making sense of the noise: Replication difficulties of Correll’s (2008) modulation of 1/f noise in a racial bias task

Abstract

Correll (Journal of Personality and Social Psychology, 94, 48-59, 2008; Study 2) found that instructions to use or avoid race information decreased the emission of 1/f noise in a weapon identification task (WIT). These results suggested that 1/f noise in racial bias tasks reflected an effortful deliberative process, providing new insights regarding the mechanisms underlying implicit racial biases. Given the potential theoretical and applied importance of understanding the psychological processes underlying implicit racial biasesand in light of the growing demand for independent direct replications of findings to ensure the cumulative nature of our science -we attempted to replicate Correll's finding in two high-powered studies. Despite considerable effort to closely duplicate all procedural and methodological details of the original study (i.e., same cover story, experimental manipulation, implicit measure task, original stimuli, task instructions, sampling frame, population, and statistical analyses), both replication attempts were unsuccessful in replicating the original finding challenging the theoretical account that 1/f noise in racial bias tasks reflects a deliberative process. However, the emission of 1/f noise did consistently emerge across samples in each of our conditions. Hence, future research is needed to clarify the psychological significance of 1/f noise in racial bias tasks.

A New Replication Norm for Psychology

“…It is most optimal for the journal that originally published an original finding to publish (unsuccessful) independent direct replication results -a standard known as the "Pottery barn rule" [93] -because this most effectively alerts readers of that journal that a finding may not be as robust as initially thought (e.g., [44] independent replications of Vess, 2012; both original and replication work published at Psychological Science). This is to be contrasted with other situations where prominent journals that published an original finding were unwilling to publish sound high-powered unsuccessful independent replications (e.g., [16,52]; see also [43]). …”

“…In the case of peer-reviewing, reviewers are typically acknowledged and thanked by journals at the end of the year and researchers will also list on their CV the journals (1) they have reviewed for and/or (2) they are on the editorial board for. For replications, researchers get credit by having their replication results published by the original journal (Pottery Barn Rule, see [93]), published at another journal (e.g., Psychonomic Bulletin & Review [52]), or at the very least get credit by posting their results to online databases that track replications (e.g., PsychFileDrawer.org; CurateScience.org).…”

“…Across all areas of psychology, there is a growing list of (prominent) findings that have not held up to independent replication attempts, including findings from cognitive psychology (retrieval- [38,17,51,18,31,57,72,71,94,44,46,52,34]). …”

“…Across all areas of psychology, there is a growing list of (prominent) findings that have not held up to independent replication attempts, including findings from cognitive psychology (retrieval-induced forgetting [54]; eye movements on recall [56]; temporal judgments [55]; protection effect [99]; mental simulation 108]; Mozart effect [95,96]), developmental psychology (synesthetic cross-modality correspondence [49]), neurophysiology (vestibular stimulation [48]), industrial/organizational psychology (utility biasing effect on selection procedures [13]), positive psychology (weather effects on life satisfaction [81]), political psychology (self-prophecy effect on voting [92]; status-legitimacy hypothesis [10]), moral psychology ("Macbeth effect" [19]), educational psychology (stereotype threat on female math performance [28,37]), color influence on exam performance [97]), evolutionary psychology (fertility on face preferences [30]; ovulation on men's testosterone [76]; sex differences in infidelity distress [35]), judgment & decision making (unconscious thought advantage [63]; choice-overload [78]), and social cognition (e.g., "social priming"/embodiment findings [38,17,51,18,31,57,72,71,94,44,46,52,34]). …”

Temporal structure in sensorimotor variability: a stable trait, but what for?

“…Nonetheless, the interest in human cognition as a critical system partly explains why the literature has mainly focused on measuring temporal structure (as opposed to manipulating it, or examining its individual differences): The interest often starts and stops at the mere existence of pink noise in the data, focusing on the ‘ubiquitousness’ of this phenomenon, as a main concern of the framework lies with generality across fields (e.g., physics, economics, biology) rather than with finding underlying neuro-cognitive processes (Wagenmakers et al, 2012). As such, temporal structure has, among other examples, been found in simple RT (Wagenmakers et al, 2004; Van Orden et al, 2003), choice RT (Kelly, Heathcote, Heath & Longstaff, 2001; Wagenmakers et al, 2004), mental rotation (Gilden et al, 2001), visual search, lexical decision, word naming, shape discrimination, and colour discrimination (Gilden, 2001; Van Orden et al, 2003), go/no-go (Simola et al, 2017), racial implicit bias tasks (Correll, 2008; Maduski & LeBel, 2015), and speech (Kello et al, 2008) – although evidence for the non-universality of temporal structure has also been found previously (see Wagenmakers et al, 2004 for an overview). Its existence is particularly clear in specific tasks, including the task used in the present article: finger tapping in synchrony with a tone (Torre et al, 2010).…”