Randomized experiments have enormous potential to improve human welfare in many domains, including healthcare, education, finance, and public policy. However, such “A/B tests” are often criticized on ethical grounds even as similar, untested interventions are implemented without objection. We find robust evidence across 16 studies of 5,873 participants from three diverse populations spanning nine domains—from healthcare to autonomous vehicle design to poverty reduction—that people frequently rate A/B tests designed to establish the comparative effectiveness of two policies or treatments as inappropriate even when universally implementing either A or B, untested, is seen as appropriate. This “A/B effect” is as strong among those with higher educational attainment and science literacy and among relevant professionals. It persists even when there is no reason to prefer A to B and even when recipients are treated unequally and randomly in all conditions (A, B, and A/B). Several remaining explanations for the effect—a belief that consent is required to impose a policy on half of a population but not on the entire population; an aversion to controlled but not to uncontrolled experiments; and a proxy form of the illusion of knowledge (according to which randomized evaluations are unnecessary because experts already do or should know “what works”)—appear to contribute to the effect, but none dominates or fully accounts for it. We conclude that rigorously evaluating policies or treatments via pragmatic randomized trials may provoke greater objection than simply implementing those same policies or treatments untested.

Self-enhancement is a positive bias in self-perception, which may imply error. However, conventional measures of self-enhancement are difference scores that do not distinguish a positive bias from a self-enhancement error, that is, they fail to identify those individuals who hold an irrationally or inaccurately positive view of themselves. We propose 2 new measures to separate error from bias. In the domain of personality judgment, we estimate a defensible bias and an enhancement error from individuals' actual and perceived similarity with others. In the domain of performance, we adapt a decision-theoretic framework to distinguish those who falsely believe to be better than average from those who actually are better. We illustrate the properties of these measures in 3 empirical studies and computer simulations. Implausibly high majorities of people consider themselves to be above average on various dimensions.

Self-enhancement bias is conventionally construed as an unwarranted social comparison in social psychology and a misperception of social reality in personality psychology. Researchers in both fields rely heavily on discrepancy scores to represent self-enhancement and fail to distinguish between a general tendency or bias to self-enhance and a self-enhancement error, or false perception of own excellence. We critically review prominent discrepancy measures and then propose a decision-theoretic alternative that [a] mitigates confounds between self-positivity and self-superiority, [b] separates error from bias, and [c] discourages reliance on measures that reify self-enhancement as a stable personality trait. To evaluate our hypotheses, we re-analyze data collected in our laboratory and perform a series of simulation studies. We share these materials with interested researchers.

Abstract. Williams and Bargh (2008) reported that holding a hot cup of coffee caused participants to judge a person’s personality as warmer and that holding a therapeutic heat pad caused participants to choose rewards for other people rather than for themselves. These experiments featured large effects ( r = .28 and .31), small sample sizes (41 and 53 participants), and barely statistically significant results. We attempted to replicate both experiments in field settings with more than triple the sample sizes (128 and 177) and double-blind procedures, but found near-zero effects ( r = −.03 and .02). In both cases, Bayesian analyses suggest there is substantially more evidence for the null hypothesis of no effect than for the original physical warmth priming hypothesis.