Certainty Is Primarily Determined by Past Performance During Concept Learning

Abstract

Prior research has yielded mixed findings on whether learners’ certainty reflects veridical probabilities from observed evidence. We compared predictions from an idealized model of learning to humans’ subjective reports of certainty during a Boolean concept-learning task in order to examine subjective certainty over the course of abstract, logical concept learning. Our analysis evaluated theoretically motivated potential predictors of certainty to determine how well each predicted participants’ subjective reports of certainty. Regression analyses that controlled for individual differences demonstrated that despite learning curves tracking the ideal learning models, reported certainty was best explained by performance rather than measures derived from a learning model. In particular, participants’ confidence was driven primarily by how well they observed themselves doing, not by idealized statistical inferences made from the data they observed.

A similar basis for judging confidence in monkeys and humans

“…This is similar to some versions of the introspection account of animal metacognition (Basile et al, 2015;Hampton, 2001;Smith et al, 2003). Although this account is quite intuitive it cannot explain a variety of phenomenon in the human metacognition literature such as the effects of past performance on confidence (Martí, Mollica, Piantadosi, & Kidd, 2018) or factors like fluency which differentially affect accuracy and confidence (Ferrigno et al, 2017;Rhodes & Castel, 2008, 2009. Inferential accounts for human metacognitive judgments.…”

“…There are a variety of additional visual fluency cues that have been shown to effect humans' confidence judgments (see Alter & Oppenheimer, 2009). Furthermore, studies have also shown that humans use many other types of (nonvisual) cues, such as conditions during testing (e.g., stimulus duration; Busey, Tunnicliff, Loftus, & Loftus, 2000), how easily an answer comes to mind (Benjamin et al, 1998), or success on previous recent trials (Martí et al, 2018). Together, the data suggest that humans and monkeys extract proxies for salience from the environment that they use as heuristics to judge the likely fidelity of their own memory and cognition.…”

The role of prior knowledge and curiosity in learning

“…Previous work shows that there are many factors that contribute to a learner's of confidence including prior performance on a task (e.g., Marti, Mollica, Piantadosi, & Kidd, 2019), and behavioral heuristics such as retrieval fluency and ease of processing within the task (e.g., Schwartz & Metcalfe, 1992;Kornell, 2009Kornell, , 2014. The relationship between confidence judgments and veridical knowledge varies greatly across domains (e.g., visual domain vs. concept learning; Barthelmé & Mamassian, 2009;Marti et al, 2019), tasks (e.g., depending on what cues are utilized), and individuals (e.g., Kruger & Dunning, 1999;Gilbert, 2015). We emphasize that we cannot definitively explain why there is an inverse relationship between curiosity and our confidencemeasuring questions in the two experiments and include these details for others to consider.…”

“Fringe” beliefs aren’t fringe

“…People have built-in mechanisms designed to help them sparsely sample information in the world to draw quick inferences and act (Kidd & Hayden, 2015;Wade & Kidd, 2019). People seek out information to reduce their uncertainty, but move from uncertain to relatively sure on the basis of heuristics like feedback (Martí, Mollica, Piantadosi, & Kidd, 2018). Once certain, people tend to stick stubbornly with their established beliefs, and it is difficult to prompt them to revise.…”