Measures of metacognitive efficiency across cognitive models of decision confidence.

Abstract

Please cite the paper as: Rausch, M., Hellmann, S., & Zehetleitner, M. ( 2023). Measures of metacognitive efficiency across cognitive models of decision confidence. Psychological Methods.

statConfR: An R Package for Static Models of Decision Confidence and Metacognition

“…Cognitive models of confidence are currently used implicitly and explicitly in a wide range of research areas in the cognitive sciences: In perception research, confidence judgments can be used to quantify perceptual sensitivity based on receiver operating characteristics (Egan et al, 1959), a method based on the signal detection rating model (Green & Swets, 1966;Hautus et al, 2021). In metacognition research, the most popular measure of metacognitive performance, the meta-d′/d′ method (Maniscalco & Lau, 2012, 2014, implicitly relies on the independent truncated Gaussian model (Rausch et al, 2023). Finally, confidence models have become a flourishing research topic in their own right (Boundy-Singer et al, 2022;Desender et al, 2021;Guggenmos, 2022;Hellmann et al, 2023Hellmann et al, , 2024Pereira et al, 2021;Rausch et al, 2018Rausch et al, , 2020Shekhar & Rahnev, 2021.…”

“…Finally, confidence models have become a flourishing research topic in their own right (Boundy-Singer et al, 2022;Desender et al, 2021;Guggenmos, 2022;Hellmann et al, 2023Hellmann et al, , 2024Pereira et al, 2021;Rausch et al, 2018Rausch et al, , 2020Shekhar & Rahnev, 2021. However, too few studies have empirically compared different confidence models (Rausch et al, 2018(Rausch et al, , 2020(Rausch et al, , 2023Shekhar & Rahnev, 2021, so there is still no consensus about the computational principles underlying confidence judgments (Rahnev et al, 2022). This is problematic because meta-d′/d′ can be biased by discrimination sensitivity, discrimination criteria, and/or confidence criteria if the generative model underlying the data is not the independent truncated Gaussian model (Rausch et al, 2023).…”

“…However, too few studies have empirically compared different confidence models (Rausch et al, 2018(Rausch et al, , 2020(Rausch et al, , 2023Shekhar & Rahnev, 2021, so there is still no consensus about the computational principles underlying confidence judgments (Rahnev et al, 2022). This is problematic because meta-d′/d′ can be biased by discrimination sensitivity, discrimination criteria, and/or confidence criteria if the generative model underlying the data is not the independent truncated Gaussian model (Rausch et al, 2023). Likewise, receiver operating characteristics in rating experiments are only appropriate measures of discrimination sensitivity if the assumptions of the signal detection rating model are correct (Green & Swets, 1966;Hautus et al, 2021).…”

“…The ReMeta toolbox provides functions for MATLAB to also fit a variety of different confidence models (Guggenmos, 2022), but some important models such as the independent truncated Gaussian R PACKAGE FOR CONFIDENCE MODELS 4 model are missing. Previous studies modelling confidence have made their analysis scripts freely available on the OSF website (Rausch et al, 2018(Rausch et al, , 2020(Rausch et al, , 2023Shekhar & Rahnev, 2021 but these analysis scripts are often tailored to specific experiments and require time and effort to adapt to new experiments. In addition, the documentation of these scripts is not always sufficient to be used without export knowledge in cognitive modelling.…”

“…The statConfR package also provides a faithful implementation of meta-d′/d′, which has been originally implemented in MATLAB (Maniscalco & Lau, 2012). Fleming provides MATLAB and R code for Hmetad, a Bayesian hierarichical version of meta-d′/d′ (Fleming, 2017), but notably the model specification of the underlying hypothetical signal detection model used for Hmetad is not identical to the one used for meta-d′/d′ (Rausch et al, 2023).…”