Eye tracking and the cognitive reflection test: Evidence for intuitive correct responding and uncertain heuristic responding

Abstract

The Cognitive Reflection Test (CRT) has been used in thousands of studies across several fields of behavioural research. The CRT has fascinated scholars because it commonly elicits incorrect answers despite most respondents possessing the necessary knowledge to reach the correct answer. Traditional interpretations of CRT performance asserted that correct responding was the result of corrective reasoning involving the inhibition and correction of the incorrect response and incorrect responding was an indication of miserly thinking without feelings of uncertainty. Recently, however, these assertions have been challenged. We extend this work by employing novel eye-tracking techniques to examine whether people use corrective cognitive pathways to reach correct solutions, and whether heuristic respondents demonstrate gaze-based signs of uncertainty. Eye movements suggest that correct responding on the CRT is the result of intuitive not corrective cognitive pathways, and that heuristic respondents show signs of gaze-based uncertainty.

Eye movements reveal that low confidence precedes deliberation

“…7. This measure was based on the previously used dwell-based measure in (Purcell et al, 2022) and in line with literature relating dwell-time to the depth with which that information is processed (e.g., Glöckner & Herbold, 2011;Rayner, 1998).…”

“…Training blocks differed from test blocks in that participants were only presented with the problem once and without a timing constraint. Participants were given feedback (correct or incorrect), presented with the full question and answer, and given a brief explanation as to why that was the correct solution (Purcell et al, 2021(Purcell et al, , 2022. For items and feedback see https://osf.io/ej3n2/.…”

“…Such studies largely employed self-reported confidence ratings (De Neys et al, 2013; Hoover & Healy, 2019; Purcell et al, 2022) and self-reported feelings of rightness (Gangemi et al, 2015). However, a recent study demonstrated that eye-tracking can be used to assess confidence on the CRT at the time of reasoning and without depending on self-report (Purcell et al, 2022). Crucially, this technique allows confidence tracking from the outset of reasoning that can be used to test the contemporary models’ proposals that lower confidence precedes deliberation.…”

Eye Movements, Pupil Dilation, and Conflict Detection in Reasoning: Exploring the Evidence for Intuitive Logic

“…Previous research has demonstrated that some forms of conflict during reasoning affects reasoner's eye movements; Ball (2014) found that participants took longer examining conflict problems even when they give belief‐based responses in error (Ball, Phillips, Wade, & Quayle, 2006). Purcell, Wastell, Howarth, Roberts, and Sweller (2022) observed that participants made a greater number of eye movements on incorrect conflict responses, compared to control items. That is, even when participants’ behavioral responses were in error, they showed intuitive sensitivity to the logical information.…”

“…That is, even when participants’ behavioral responses were in error, they showed intuitive sensitivity to the logical information. Moreover, recent studies reveal that eye movements are negatively related to judgments of confidence (Purcell et al., 2022) and positively related to the engagement of effortful thinking (Purcell, Wastell, & Sweller, 2022). The growing evidence suggesting that metacognitions and conflict have systematic impacts on eye movements is promising for the use of gaze‐based indicators in determining the intuitive impacts of belief–logic conflict.…”