Teachers’ monitoring of students’ text comprehension: can students’ keywords and summaries improve teachers’ judgment accuracy?

Abstract

We investigated intra-individual monitoring and regulation in learning from text in sixth-grade students and their teachers. In Experiment 1, students provided judgments of learning (JOLs) for six texts in one of three cue-prompt conditions (after writing delayed keywords or summaries or without a cue prompt) and then selected texts for restudy. Teachers also judged their students’ learning for each text, while seeing - if present - the keywords or summaries each student had written for each text, and also selected texts for restudy. Overall, monitoring accuracy was low (.10 for students, −.02 for teachers) and did not differ between cue-prompt conditions. Regulation, indexed by the correlation between JOLs and restudy selections, was significant (−.38 for students, −.60 for teachers), but was also not affected by cue-prompt condition. In Experiment 2, teachers judged students’ comprehension of six texts without knowing the students’ names, so that only the keywords and summaries, not prior impressions, could inform judgments. Again, monitoring accuracy was generally low (.06), but higher for keywords (.23) than for summaries (−.10). These results suggest that monitoring intra-individual differences in students’ learning is challenging for teachers. Analyses of the diagnosticity and utilization of keywords suggest that these may contain insufficient cues for improving teacher judgments at this level of specificity.

Development of Children’s monitoring and control when learning from texts: effects of age and test format

“…Furthermore, even though correlations between performance on open-ended and true-false questions were moderate to high, indicating that both test types actually measured text comprehension, internal consistency of the open-ended as well as true-false questions was rather low, especially for true-false questions. Such low internal consistency has also been observed in other research assessing text comprehension for different texts (e.g., Engelen et al 2018). Because children read six different texts, it could be that each child's understanding of the different texts varied, thus resulting in higher performance on questions for certain texts and lower performance for other texts.…”

“…Internal consistency of the true-false questions is rather low, however, low internal consistency is not uncommon for text comprehension tests (cf. Engelen et al 2018;Van Loon et al 2015). It lies in the very nature of metacognitive research that questions for which monitoring judgments are gathered, cover the entire variation of difficulty to allow tapping the whole range of monitoring processes.…”

To What Extent Do Situation-Model-Approach Interventions Improve Relative Metacomprehension Accuracy? Meta-Analytic Insights

“…The variance of this composite score was computed based on the variance of each effect size and the correlation between the two effects (which was set to r = .50 given the equal sample size in conditions; see, e.g., Borenstein et al 2009). The same issue applied to Experiment 1 by Engelen et al (2018), which compared delayed-summary writing and delayed-keywords listing to the same control group. Again, we calculated a composite score that was the mean of the effect for delayed-summary writing versus control and the effect for delayed-keywords listing versus control.…”

“…As already stated, based on Experiment 4 by Griffin et al (2019b), effect sizes for selfexplaining and setting a comprehension-test expectancy were calculated using the same control group. Likewise, based on Experiment 1 by Engelen et al (2018), effect sizes for delayed-summary writing and delayed-keywords listing were computed using the same control group. In the moderator analyses, the two effects from one study were treated as independent effects so that they could be included in the study pool for both interventions.…”

The keyword effect: A conceptual replication, effects on bias, and an optimization

“…with both high school and university students and with different learning materials (e.g., De Bruin et al 2011;Thiede et al 2005Thiede et al , 2017; but see Engelen et al 2018). Despite these promising results, however, there are still some important open issues that need to be addressed.…”