A Multi-Site Collaborative Study of the Hostile Priming Effect

Abstract

In a now-classic study by Srull and Wyer (1979), people who were exposed to phrases with hostile content subsequently judged a man as being more hostile. And this “hostile priming effect” has had a significant influence on the field of social cognition over the subsequent decades. However, a recent multi-lab collaborative study (McCarthy et al., 2018) that closely followed the methods described by Srull and Wyer (1979) found a hostile priming effect that was nearly zero, which casts doubt on whether these methods reliably produce an effect. To address some limitations with McCarthy et al. (2018), the current multi-site collaborative study included data collected from 29 labs. Each lab conducted a close replication (total N = 2,123) and a conceptual replication (total N = 2,579) of Srull and Wyer’s methods. The hostile priming effect for both the close replication (d = 0.09, 95% CI [-0.04, 0.22], z = 1.34, p = .16) and the conceptual replication (d = 0.05, 95% CI [-0.04, 0.15], z = 1.15, p = .58) were not significantly different from zero and, if the true effects are non-zero, were smaller than what most labs could feasibly and routinely detect. Despite our best efforts to produce favorable conditions for the effect to emerge, we did not detect a hostile priming effect. We suggest that researchers should not invest more resources into trying to detect a hostile priming effect using methods like those described in Srull and Wyer (1979).

A Review of Multisite Replication Projects in Social Psychology: Is It Viable to Sustain Any Confidence in Social Psychology’s Knowledge Base?

“…McCarthy et al (2021) ran both a close replication and a conceptual replication protocol. Their preregistered criteria dictated deleting 34% and 35.6% of participants, respectively, and they excluded significantly more in the hostile priming condition than in the neutral control condition, which threatens the internal validity of their findings.…”

“…To summarize: excluding a substantial number of participants sometimes improved results (Bouwmeester et al, 2017; Dang et al, 2021; O’Donnell et al, 2018), sometimes made them substantially worse (Baranski et al, 2020; Moran et al, 2021; Vohs et al, 2021), and sometimes made no difference (McCarthy et al, 2021; McCarthy, Hartnett, et al, 2018; Sanchez et al, 2017; Skorb et al, 2020; Verschuere et al, 2018). Apart from Dang et al (2021), whose results were significant in both analyses, and McCarthy et al (2021), whose results were nonsignificant either way, many of the exclusions were decisive in terms of whether the replication’s omnibus finding was significant or not. The purpose of excluding participants is presumably to improve the test of the hypothesis and, if the hypothesis is correct, to increase the chances of a significant result.…”

“…If exclusion rates are high, analyses should be reported both ways (i.e., both full sample and after exclusions). As a commendable example, McCarthy et al (2021) reported their preregistered analyses but also, after noting the high proportion of discarded data, reported multiple analyses with different exclusion criteria (as well as no exclusion criteria).…”

Bridging the Gap between Spontaneous Behavior- and Stereotype-based Impressions

“…We observe that, in the previously described approach, any ambiguity is essentially pre-tested out of the stimuli. Behaviors can be ambiguous with respect to their implications; sometimes a behavior could be potentially friendly or unfriendly depending on numerous factors (e.g., the classic "Donald" paragraph featuring ambiguously hostile behaviors; Srull & Wyer, 1979; see also McCarthy et al, 2018McCarthy et al, , 2021. Because Uleman et al's original research was aimed at establishing that spontaneous trait inferences occur, it became paradigmatic to focus on clearly trait-implying behaviors.…”

Seeking Solace, Not Catharsis: Frustration Diminishes the Inclination to Play Violent Video Games

“…Methodological concerns (e.g., Elson et al, 2015;Hilgard et al, 2017a;Hilgard et al, 2017b) and theoretical issues (e.g., Giolla et al, 2022;McCarthy et al, 2021) have resulted in a notable lack of consensus in the field (Mehta & Lakens, 2023). In contrast, studies on the effects of long-term exposure to VVGs have relied on larger and more relevant samples, such as habitual gamers, and have utilized less flexible, more reliable and rigorously validated measures, which lends greater credibility to the findings (Flake & Fried, 2020).…”