Accuracy and stability of metacognitive monitoring: A new measure

Cheng, Chao-Ming

doi:10.3758/brm.42.3.715

Cited by 9 publications

(4 citation statements)

References 28 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Also, more particularly, we would like to know the relative effects on recall that are attributable to restudy of not just selected items but of unselected items as well. The restudy selec- 1 The gamma correlation has traditionally been the preferred measure of relative accuracy, following an analysis reported by Nelson (1984), although there have been recent criticisms of gamma as a measure of relative accuracy, and several alternative measures have been proposed (see Benjamin & Diaz, 2008;Cheng, 2010;Masson & Rotello, 2009;Spellman, Bloomfield, & Bjork, 2008). We continue to use gamma in this article despite potential shortcomings because there is no consensus as yet to supplant it, and using it allows comparison to earlier research.…”

Section: Limitations Of Previous Studiesmentioning

confidence: 99%

Does delaying judgments of learning really improve the efficacy of study decisions? Not so much.

Kimball

Smith

Muntean

2012

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

A widely held assumption in metamemory is that better, more accurate metamemory monitoring leads to better, more efficacious restudy decisions, reflected in better memory performance--we refer to this causal chain as the restudy selectivity hypothesis. In 3 sets of experiments, we tested this hypothesis by factorially manipulating metamemory monitoring accuracy and self-regulation of study. To manipulate monitoring accuracy, we compared judgments of learning (JOLs) made contemporaneously with a delayed retrieval attempt to JOLs either made at a delay without attempting retrieval or made immediately after study; in previous studies, delayed retrieval-based JOLs have robustly predicted recall with greater relative accuracy than have the other JOL types. To manipulate self-regulation of study, in Experiments 1A-1C and 2A-2C, we compared conditions in which participants' restudy selections were honored with conditions in which they were completely or randomly dishonored; in Experiments 3A-3C, we randomly honored or dishonored half of the restudy selections and half of the nonselections. Results revealed that the benefit of delayed, retrieval-based JOLs for final memory performance was due largely to the selection of more items for restudy rather than to better discriminations between items that would benefit more versus less from restudy. In most cases, gains in recall due to greater self-regulation of study did not increase with better monitoring accuracy; when they did, the effect was extremely small. The surprising conclusion was that restudy decisions were not very much more efficacious under conditions that yield greater monitoring accuracy than under those that do not.

show abstract

Section: Limitations Of Previous Studiesmentioning

confidence: 99%

Does delaying judgments of learning really improve the efficacy of study decisions? Not so much.

Kimball

Smith

Muntean

2012

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

show abstract

“…Although this measure has been criticized as a measure of metamemory accuracy (Cheng, 2010;Nelson, 1996;Wright, 1996), it is sometimes used as a supplement of G (e.g., Reggev, Zuckerman, & Maril, 2011). Another measure of metacognitive accuracy is Hart's difference score, D, which is calculated as the difference in mean metacognitive judgments between recalled and unrecalled items (Hart, 1965; this is also conceptually equivalent to corrected hit rate in the case where both criterion and predictor variables are dichotomous).…”

Section: Correlation Familymentioning

confidence: 99%

“…HC can be computed when both predictor and criterion variables are dichotomous using the following formula (Romesburg, 1984): where N T is the total number of items, N C is the number of correct judgments (the number of recalled items that are judged to be recalled and the number of forgotten items that are judged to be forgotten), and N IC is the number of incorrect judgments (the number of recalled items that are judged to be forgotten and the number of forgotten items that are judged to be recalled). Although this measure has been criticized as a measure of metamemory accuracy (Cheng, 2010; Nelson, 1996; Wright, 1996), it is sometimes used as a supplement of G (e.g., Reggev, Zuckerman, & Maril, 2011). Another measure of metacognitive accuracy is Hart’s difference score, D , which is calculated as the difference in mean metacognitive judgments between recalled and unrecalled items (Hart, 1965; this is also conceptually equivalent to corrected hit rate in the case where both criterion and predictor variables are dichotomous).…”

Section: Measuring Relative Accuracy Of Metamemory Monitoringmentioning

confidence: 99%

Type I error inflation in the traditional by-participant analysis to metamemory accuracy: A generalized mixed-effects model perspective.

Murayama

Sakaki

Yan

et al. 2014

Journal of Experimental Psychology: Learning, Memory, and Cogni

109

105

View full text Add to dashboard Cite

In order to examine metacognitive accuracy (i.e., the relationship between metacognitive judgment and memory performance), researchers often rely on by-participant analysis, where metacognitive accuracy (e.g., resolution, as measured by the gamma coefficient or signal detection measures) is computed for each participant and the computed values are entered into group-level statistical tests such as the t test. In the current work, we argue that the by-participant analysis, regardless of the accuracy measurements used, would produce a substantial inflation of Type I error rates when a random item effect is present. A mixed-effects model is proposed as a way to effectively address the issue, and our simulation studies examining Type I error rates indeed showed superior performance of mixed-effects model analysis as compared to the conventional by-participant analysis. We also present real data applications to illustrate further strengths of mixed-effects model analysis. Our findings imply that caution is needed when using the by-participant analysis, and recommend the mixed-effects model analysis.

show abstract

“…Outra maneira de se avaliar o monitoramento metacognitivo é a partir do julgamento metacognitivo (Cheng, 2010;Efklides, 2006;Nelson & Narens, 1994;Schelini et al, 2016;Son & Schwartz, 2002), que se refere ao julgamento probabilístico, uma reflexão do sujeito sobre sua performance (JoPs) antes, durante ou depois de uma tarefa (Fleming, Massoni, Gajdos & Vergnaud, 2016;Schraw, 2009). Os julgamentos prospectivos são aqueles emitidos antes do início ou durante a tarefa, e são estimativas do indivíduo sobre sua atuação a ser apresentada.…”

Section: Introductionunclassified

Avaliação de monitoramento metacognitivo a partir do julgamento de desempenho em adultos em processo de alfabetização

Fujie

Schelini

2019

Estud. Interdiscip. Psicol.

View full text Add to dashboard Cite

A metacognição, a capacidade do indivíduo em autorregular seus processos cognitivos, engloba o monitoramento metacognitivo, pelo qual o sujeito acompanha e avalia seu desempenho cognitivo. O presente estudo teve como finalidade investigar o monitoramento metacognitivo de adultos em processo de alfabetização por meio do julgamento de desempenho em tarefas cognitivas, assim como observar possíveis correlações entre desempenho real e estimado dos participantes. Participaram do estudo 15 adultos de um programa de alfabetização, de ambos os gêneros e com idades entre 20 e 74 anos. Os materiais aplicados foram: Entrevista Inicial; Teste R-1 e subtestes Código, Procurar Símbolos e Dígitos do WAIS III; Registro de Julgamentos de 0 a 10 e Registro de Julgamento a partir de um objeto de medida. Resultados indicaram correlação significativa entre os escores brutos de desempenho real dos subtestes Códigos, Procurar Símbolos e Dígitos com seus julgamentos de desempenho a partir de um objeto de medida.

show abstract

Accuracy and stability of metacognitive monitoring: A new measure

Cited by 9 publications

References 28 publications

Does delaying judgments of learning really improve the efficacy of study decisions? Not so much.

Does delaying judgments of learning really improve the efficacy of study decisions? Not so much.

Type I error inflation in the traditional by-participant analysis to metamemory accuracy: A generalized mixed-effects model perspective.

Avaliação de monitoramento metacognitivo a partir do julgamento de desempenho em adultos em processo de alfabetização

Contact Info

Product

Resources

About