Why does the Cognitive Reflection Test (sometimes) predict utilitarian moral judgment (and other things)?

Abstract

The Cognitive Reflection Test (CRT) is thought to correlate with measures of utilitarian moral judgment because it measures system-2 correction of an initial intuitive response. And some theories of moral judgment hold that the same thing often happens when people arrive a utilitarian judgments. We find, however, that CRT-type items (using logic as well as arithmetic) can work just as well when they do not have obvious intuitive answers at predicting utilitarian moral judgment, assessed with self-report questionnaires as well as with hypothetical scenarios, and also at predicting a measure of actively openminded thinking (AOT). Moreover, long response times, as well as high accuracy, also predict moral judgment and other outcomes. The CRT might thus be considered a test of reflection-impulsivity (RI). However, RI is only part of AOT, because RI is concerned only with the amount of thinking, not its direction. Tests of AOT also predict utilitarian moral judgments. Individual differences in AOT and moral judgments are both strongly (negatively) associated with belief that morality comes from God and cannot be understood through thought. The correlation of CRT and utilitarian judgment, when found, is thus likely due to the (imperfect) correlation of AOT and CRT. Intuition in these domains is thus not necessarily something that people overcome through additional thinking, but rather what they rely on when they do not think very much.

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics.

“…Inductive pattern recognition was assessed by a short form of the Ravens Advanced Progressive Matrices (Ravens APM; Bors & Stokes, 1998), which circumvents cultural or linguistic knowledge by testing spatial problem-solving skills. Cognitive control was measured by the three-item Cognitive Reflection Test (CRT; Frederick, 2005) and the four-item extension of the CRT (Baron, Scott, Fincher, & Metz, 2014), with questions such as, "All flowers have petals. Roses have petals.…”

Analytic atheism: A cross-culturally weak and fickle phenomenon?

“…The CRT is widely used, but may not measure the most relevant sort of reflection very well (Baron, Scott, Fincher, & Metz, 2015). Similar and convergent measures of analytic thinking and cognitive reflection also similarly predict religious disbelief (Pennycook, Cheyne, Barr, Koehler & Fugelsang, 2014a;Saribay & Yilmaz, 2017), bolstering the claims to generalizability across measures and also, possibly, providing more direct measures of the relevant traits.…”

“…Cognitive reflection differentially predicts different facets of religiosity (Bahçekapili & Yilmaz, 2017), as well as religious affiliation (Pennycook et al, 2012) and other related constructs (Pennycook, Cheyne, Barr, Koehler & Fugelsang, 2014b;Saribay & Yilmaz, 2017;Yilmaz & Saribay, 2016). And other measures of reflection, as well as CRT items, predict specific religious beliefs such as endorsement of "divine command theory " Piazza & Landy, 2013;Baron et al, 2015), a view that explicitly discourages reflection on the ground that the word of God is beyond human understanding. The present paper can serve as a jumping board for additional cross-cultural exploration.…”

Explaining sex differences on the Cognitive Reflection Test

“…The correct answer is 5 cents. Although the CRT is correlated with cognitive ability and numeracy, research suggests that it explains incremental variance in outcomes, including performance on various judgment and decision tasks (Baron, Scott, Fincher, & Metz, 2015;Liberali, Reyna, Furlan, Stein, & Pardo, 2011;Toplak, West, & Stanovich, 2011) as well as a (dis)belief in God (Shenhav, Rand, & Greene, 2011). Toplak et al (2011) concluded that the CRT is "a particularly potent measure of the tendency toward miserly processing because it is a performance measure rather than a self-report measure" (p. 1275).…”