Measures of relative metacognitive accuracy are confounded with task performance in tasks that permit guessing

Abstract

This article investigates the concern that assessment of metacognitive resolution (or relative accuracy—often evaluated by gamma correlations or signal detection theoretic measures such as da) is vulnerable to an artifact due to guessing that differentially impacts low as compared to high performers on tasks that involve multiple-choice testing. Metacognitive resolution refers to people’s ability to tell, via confidence judgments, their correct answers apart from incorrect answers, and is theorized to be an important factor in learning. Resolution—the trial-by-trial association between response accuracy and confidence in that response’s accuracy—is a distinct ability from knowledge, or accuracy, and instead indicates a higher-order self-evaluation. It is therefore important that measures of resolution are independent of domain-knowledge accuracy. We conducted six experiments that revealed a positive correlation between metacognitive resolution and performance in multiple-choice mathematics testing. Monte Carlo simulations indicated, however, that resolution metrics are increasingly negatively biased with decreasing performance, because multiple-choice tasks permit correct guessing. We, therefore, argue that the observed positive correlations were probably attributable to an artifact rather than a true correlation between psychological abilities. A final experiment supported the guessing-related confound hypothesis: Resolution and performance were positively correlated in multiple-choice testing, but not in free-response testing. This study brings to light a previously underappreciated limitation in assessing metacognitive resolution and its relation to task performance in criterion tasks that may involve guessing.

Metacognitive Strategies for Developing Complex Geographical Causal Structures—An Interventional Study in the Geography Classroom

“…As a consequence of the study, it can be assumed that metacognitive control and monitoring processes raise the task-related and topical accuracy. This confirms findings by Vuorre and Metcalfe [81] (p. 19) and Dunlovsky et al [74] (pp. 23-27) on the connection of metacognitive competences and the precision or matching of the task processing or learning success, respectively.…”

“…The metacognitive knowledge and strategies are to be activated, organized, and verbalized by means of an action plan and thus be made available for the task-solving process [93] (pp. [79][80][81][82][83]. The method's targets are increasing problem comprehension, activating prior knowledge, and bringing to mind action steps during the problem-solving process.…”

“…During the process of metacognition, these heuristics get activated more easily and repeatedly adjusted in relation to the problem’s demands (multicausal links) in connection with the technical and linguistic resources available to the students [ 18 ] (p. 908) ( Table 2 ). They might, as a consequence, be more capable of monitoring the development of complex multicausal links and of verifying or rectifying the result, i.e., the linguistically correct representation of (geographical) content relationships [ 81 ] ( Table 2 ). So far, it has remained mostly unexplored how the triad effective in the classroom, consisting of motivation, cognition, and metacognition [ 83 ], ought to be designed in order to successfully develop complex geographical causal structures in the geography lesson.…”

A Meta-Analysis Investigating the Association Between Metacognition and Math Performance in Adolescence

“…This phenomenon may be dependent on whether the MC measure being used is clearly integrated with the material being learned. Vuorre and Metcalfe (2021) noted that if academic tests are used that permit guessing (e.g. multiple-choice tests), for example, then "metacognitive misses" (i.e.…”

The role of metacognition in monitoring performance and regulating learning in early readers

“…However, other studies reported no association between metacognitive monitoring and cognitive ability such as memory strategies (Kelly et al, 1976), text comprehension skills (Griffin et al, 2009). It has been suggested that these results may be attributed to the use of metacognitive indexes which are susceptible to the confounding effects of confidence bias, participants' level of type-1 performance and methods that permit guessing, which can differentially affect the estimation of metacognitive accuracy in high vs poor performers (Vuorre & Metcalfe, 2021). In the present study, we used a bias free signal detection theoretic framework,…”