Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of thirteen classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, ten effects replicated consistently.One effect -imagined contact reducing prejudice -showed weak support for replicability. And two effects -flag priming influencing conservatism and currency priming influencing system justification -did not replicate. We compared whether the conditions such as lab versus online or U.S. versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect. Word Count = 121 words Many Labs 3 Investigating variation in replicability: A "Many Labs" Replication ProjectReplication is a central tenet of science; its purpose is to confirm the accuracy of empirical findings, clarify the conditions under which an effect can be observed, and estimate the true effect size (Brandt et al., 2013; Open Science Collaboration, 2012. Successful replication of an experiment requires the recreation of the essential conditions of the initial experiment. This is often easier said than done. There may be an enormous number of variables influencing experimental results, and yet only a few tested. In the behavioral sciences, many effects have been observed in one cultural context, but not observed in others. Likewise, individuals within the same society, or even the same individual at different times (Bodenhausen, 1990), may differ in ways that moderate any particular result.Direct replication is infrequent, resulting in a published literature that sustains spurious findings (Ioannidis, 2005) and a lack of identification of the eliciting conditions for an effect. While there are good epistemological reasons for assuming that observed phenomena generalize across individuals and contexts in the absence of contrary evidence, the failure to directly replicate findings is problematic for theoretical and practical reasons. Failure to identify moderators and boundary conditions of an effect may result in overly broad generalizations of true effects across situations (Cesario, 2013) or across individuals (Henrich, Heine, & Norenzayan, 2010). Similarly, overgeneralization may lead observations made under laboratory observations to be inappropriately extended to ecological contexts that differ in important ways (Henry, MacLeod, Phillips, & Crawford, 2004). Practically, attempts to closely replicate research findings can reveal important differences in what is considered a direct replication (Schimdt, 2009), thus leading to refinements of the initial theory (e.g., Aronson, 1992, Greenwald et al., 1986. Close replication can also lead to Many Labs 4 the clarification of tacit methodological knowledge that is necessary to elicit the effect of interest (Collins,...

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance ( p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion ( p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.

We conducted preregistered replications of 28 classic and contemporary published findings with protocols that were peer reviewed in advance to examine variation in effect magnitudes across sample and setting. Each protocol was administered to approximately half of 125 samples and 15,305 total participants from 36 countries and territories. Using conventional statistical significance (p &lt; .05), fifteen (54%) of the replications provided evidence in the same direction and statistically significant as the original finding. With a strict significance criterion (p &lt; .0001), fourteen (50%) provide such evidence reflecting the extremely high powered design. Seven (25%) of the replications had effect sizes larger than the original finding and 21 (75%) had effect sizes smaller than the original finding. The median comparable Cohen’s d effect sizes for original findings was 0.60 and for replications was 0.15. Sixteen replications (57%) had small effect sizes (&lt; .20) and 9 (32%) were in the opposite direction from the original finding. Across settings, 11 (39%) showed significant heterogeneity using the Q statistic and most of those were among the findings eliciting the largest overall effect sizes; only one effect that was near zero in the aggregate showed significant heterogeneity. Only one effect showed a Tau &gt; 0.20 indicating moderate heterogeneity. Nine others had a Tau near or slightly above 0.10 indicating slight heterogeneity. In moderation tests, very little heterogeneity was attributable to task order, administration in lab versus online, and exploratory WEIRD versus less WEIRD culture comparisons. Cumulatively, variability in observed effect sizes was more attributable to the effect being studied than the sample or setting in which it was studied.

Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of thirteen classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, ten effects replicated consistently. One effect – imagined contact reducing prejudice – showed weak support for replicability. And two effects – flag priming influencing conservatism and currency priming influencing system justification – did not replicate. We compared whether the conditions such as lab versus online or U.S. versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect.

The relationships between subjective status and perceived legitimacy are important for understanding the extent to which people with low status are complicit in their oppression. We use novel data from 66 samples and 30 countries ( N = 12,788) and find that people with higher status see the social system as more legitimate than those with lower status, but there is variation across people and countries. The association between subjective status and perceived legitimacy was never negative at any levels of eight moderator variables, although the positive association was sometimes reduced. Although not always consistent with hypotheses, group identification, self‐esteem, and beliefs in social mobility were all associated with perceived legitimacy among people who have low subjective status. These findings enrich our understanding of the relationship between social status and legitimacy.

Implicit theories of intelligence have been proposed to predict a large number of different outcomes in education. The belief that intelligence is malleable (growth mindset) is supposed to lead to better academic achievement and students' mindset is therefore a potential target for interventions. The present study used a large sample of university applicants (N = 5,653) taking a scholastic aptitude test to further examine the relationship between mindset and achievement in the academic domain. We found that results in the test were slightly negatively associated with growth mindset (r = -.03). Mindset showed no relationship with the number of test administrations participants signed up for and it did not predict change in the test results. The results show that the strength of the association between academic achievement and mindset might be weaker than previously thought. Keywords mindset, scholastic aptitude, achievement, implicit theories, education, intelligence, selfregulation, performanceIntelligence predicts a large number of important life outcomes (Neisser et al., 1996). Apart from the level of intelligence itself, people may also differ in their implicit theories about the nature of intelligence and its malleability. On one end of the continuum, some people consider intelligence fixed and unchangeable. On the opposite side, some people believe that intelligence can be improved and developed. These two views are labeled as a fixed mindset and growth mindset, respectively (Dweck, 2012;Dweck & Leggett, 1988;Molden & Dweck, 2006). A recent metaanalysis showed that implicit theories about intelligence are associated with various aspects of self-regulation (Burnette, O'Boyle, VanEpps, Pollack, & Finkel, 2013). People with growth mindset tend to set learning goals rather than performance goals, they tend to adopt strategies focused on mastery rather than helpless-oriented strategies, and when monitoring their goal achievement, they tend to focus on future expectations of success rather than negative emotions. Importantly, the self-regulatory processes associated with growth mindset are positively associated with goal achievement. Growth mindset itself was therefore shown to be associated with a higher likelihood of achieving one's goals; even though, the results of the meta-analysis suggest that the association is weak (Burnette et al., 2013).While Burnette et al. (2013) showed the association between implicit theories and achievement, some research (e.g., Blackwell, Trzesniewski, & Dweck, 2007; see also Yeager et al., 2014) suggests that people with growth and fixed mindset may not differ in their baseline abilities and the difference appears only when encountering adversities or challenging situations. For example, students with growth and fixed mindset may not differ in their mathematics grades before entering junior high school, but the transition to high school poses a significant challenge to which students with growth mindset might adapt better. As a results, mathematics grades were shown to st...

Beliefs about polarization have significant social consequences, whether accurate or not [1][2][3][4] . They also complicate the study of social issues as reported attitudes might be impacted by inaccurate out-group perceptions and thus make conflicts around a specific issue or policy appear more severe than they actually are 5 . This can result in significant distortions in numerous behaviours, including health, voting and financial choices, each of which has consequences on population wellbeing 6 .There is growing interest in the origins of polarization across populations as well as its features and impacts across communities and society 7,8 . This interest is global and includes both scientific research as well as general public interest 9,10 . As polarization seemingly permeates a growing number of personal and public domains, there is some sense of renewed urgency to understand it and its effects more deeply 11,12 . This includes extending study to understand the extent of polarization on community and global levels.However, with increasing interest in polarization itself, broadening evidence indicates that inaccuracies in perceptions of how the out-group feels about the in-group can be harmful 13 . The origin of these 'meta-perceptions' may be rooted in negative stereotypes that individuals feel have been applied to them, often incorrectly 14 . This results in an inaccurate perception of differences in beliefs and attitudes between groups 15 , which can have negative results for individuals 16,17 . On a population level, such misperceptions can even result in overstated reactions that exacerbate existing biases 18 .To investigate roots and moderators of polarization across groups, Lees and Cikara 13 ran a series of experiments with US participants identifying as Republican or Democrat. In what they refer

The COVID-19 pandemic has increased negative emotions and decreased positive emotions globally. Left unchecked, these emotional changes might have a wide array of adverse impacts. To reduce negative emotions and increase positive emotions, we tested the effectiveness of reappraisal, an emotion-regulation strategy that modifies how one thinks about a situation. Participants from 87 countries and regions (n = 21,644) were randomly assigned to one of two brief reappraisal interventions (reconstrual or repurposing) or one of two control conditions (active or passive). Results revealed that both reappraisal interventions (vesus both control conditions) consistently reduced negative emotions and increased positive emotions across different measures. Reconstrual and repurposing interventions had similar effects. Importantly, planned exploratory analyses indicated that reappraisal interventions did not reduce intentions to practice preventive health behaviours. The findings demonstrate the viability of creating scalable, low-cost interventions for use around the world.