What's Context Got to Do with It? Comparative Difficulty of Test Questions Influences Metacognition and Corrected Scores for Formula‐scored Exams

Arnold, Michelle M.; Graham, Kristin M.; Hollingworth-Hughes, Sinead

doi:10.1002/acp.3312

Cited by 6 publications

(7 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contrary to expectations, immediate feedback did not improve monitoring accuracy; monitoring accuracy was low with or without immediate feedback, and monitoring accuracy with immediate feedback was significantly less accurate than with no feedback (IF < NF, Cohen's d = .62, p = .012). A non-monotonic relationship between item performance and confidence was observed (hard-easy effect, Arnold et al, 2017;Juslin et al, 2000) that attenuates the findings here for relative monitoring accuracy (see Figure 2).…”

Section: Discussioncontrasting

confidence: 55%

“…A premise of this investigation and perhaps of all SRL investigations is that of a monotonic relationship between the observed difficulty of the lesson content based on actual performance scores and the learner's judgement of the difficulty of that content -that difficult material is perceived to be difficult and easy material is perceived to be easy. But previous SRL research shows that students tend to be overconfident on difficult things and under confident on easy things, the hard-easy effect (Arnold, Graham, & Hollingworth-Hughes, 2017;Juslin, Winman, & Olsson, 2000). So what is perceived difficulty?…”

Section: Hard-easy Effectmentioning

confidence: 95%

See 1 more Smart Citation

Item-level monitoring, response style stability, and the hard-easy effect

Clariana

Park

2021

Education Tech Research Dev

View full text Add to dashboard Cite

Cognitive and metacognitive processes during learning depend on accurate monitoring, this investigation examines the influence of immediate item-level knowledge of correct response feedback on cognition monitoring accuracy. In an optional end-of-course computer-based review lesson, participants (n = 68) were randomly assigned to groups to receive either immediate item-by-item feedback (IF) or no immediate feedback (NF).Item-by-item monitoring consisted of confidence self-reports. Two days later, participants completed a retention test (IF = NF, no significant difference). Monitoring accuracy during the review lesson was low, and contrary to expectations was significantly less with immediate feedback (IF < NF, Cohen's d = .62). Descriptive data shows that (1) monitoring accuracy can be attributed to cues beyond actual item difficulty, (2) a hard-easy effect was observed where item difficulty was related to confidence judgements as a non-monotonic function, (3) response confidence was predicted by the Coh-Metrix dimension Word Concreteness in both the IF and NF treatments, and (4) significant autocorrelations (hysteresis) for confidence measures were observed for NF but not for IF. It seems likely that monitoring is based on multiple and sometimes competing cues, the salience of each relates in some degree to content difficulty, but that the stability of individual response styles plays a substantive role in monitoring. This investigation shows the need for new applications of technology for monitoring multiple measures on the fly to better understand SRL processes to support all learners.

show abstract

Section: Discussioncontrasting

confidence: 55%

Section: Hard-easy Effectmentioning

confidence: 95%

Item-level monitoring, response style stability, and the hard-easy effect

Clariana

Park

2021

Education Tech Research Dev

View full text Add to dashboard Cite

show abstract

“…However, from previous studies found that the test models developed in science tend to only focus on the aspects of knowledge, especially facts (M. M. Arnold, Graham, & Hollingworth-Hughes, 2017). In addition, the types of questions developed do not support the learning of science using the inquiry approach in the classroom.…”

Section: Introductionmentioning

confidence: 94%

The Factors Affecting Teachers’ Readiness In Developing Science Concept Assessment Through Inquiry-Based Learning Process In Elementary Schools

Budiastra

Erlina

Wicaksono

2019

ASSRJ

View full text Add to dashboard Cite

Teachers need to develop their professionalism in teaching to achieve 21st century learning competencies. The role of the teacher is not only to provide information but also to assess the learning process and outcomes comprehensively and in an integrated manner. This study aimed to describe the factors that support the readiness of teachers to carry out assessments in inquiry-based science in elementary schools. Respondents in this study were 87 elementary school teachers who were 87 students spread across four regions of the UT Regional Office namely Pangkalpinang, Jakarta, Denpasar, and Kupang. These four regions of the UT Regional Office were chosen as a representation of the regions of West, Central and East Indonesia. From this sample, teachers were examined to see their readiness to compose a lesson plan and carry out an inquiry assessment. Next, teachers filled in the response sheet to find out their readiness to carry out an inquiry assessment in science learning in elementary schools. Teaching inquiry-based science required teachers to spend more time preparing lesson plans to prepare tools and learning materials. Factors which supported the readiness of teachers to measure the inquiry-based science learning process in elementary schools, including their sufficient capacity to develop lesson plans which were oriented to the inquiry process, peer support and teachers’ working group. Some efforts were done to overcome obstacles namely developing the ability to assess the mastery of science concepts through inquiry-based learning processes in elementary schools, utilizing teachers’ working group (TWG) and the support of school principals/assessors to make relevant assessment tools.

show abstract

“…Various factors, such as experience ( Cichoń et al 2018 ), feedback ( Iida et al 2020 ), and effort ( Lee and Daunizeau 2021 ) can affect individuals’ confidence judgments. The most important and common influencing factor is the item difficulty ( Arnold et al 2017 ; Clariana and Park 2021 ). The item difficulty significantly impacts the accuracy and confidence level of individual judgments, leading to a “hard-easy effect” on the accuracy of confidence judgments.…”

Section: Introductionmentioning

confidence: 99%

The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model

Zhou

Jia

2023

J. Intell.

View full text Add to dashboard Cite

The factors that influence metacognitive judgments often appear in combination, rather than in isolation. The multi-cue utilization model proposes that individuals often make use of multiple cues when making judgments. Previous studies have focused on the integration of intrinsic and extrinsic cues, while the current investigation examines the integration and influence of intrinsic cues and mnemonic cues. Judgment of confidence is a common form of metacognitive judgment. In this study, 37 college students completed Raven’s Progressive Matrices and made judgments of confidence. We used the cross-level moderated mediation model to explore the impact of item difficulty on confidence judgments. Our results indicated that item difficulty negatively predicts the level of confidence. Item difficulty has an impact on the confidence evaluation by altering the processing fluency of intermediate variables. The joint effect of intrinsic cue item difficulty and mnemonic cue processing fluency influences confidence judgments. Additionally, we found that intelligence moderates the effect of difficulty on processing fluency across levels. Specifically, individuals with higher intelligence exhibited lower fluency on difficult tasks and higher fluency on simple tasks than individuals with lower intelligence. These findings expand on the multi-cue utilization model and integrate the influence mechanism of intrinsic and mnemonic cues on confidence judgments. Finally, we propose and verify a cross-level moderated mediation model that explains how item difficulty affects confidence judgments.

show abstract

What's Context Got to Do with It? Comparative Difficulty of Test Questions Influences Metacognition and Corrected Scores for Formula‐scored Exams

Cited by 6 publications

References 46 publications

Item-level monitoring, response style stability, and the hard-easy effect

Item-level monitoring, response style stability, and the hard-easy effect

The Factors Affecting Teachers’ Readiness In Developing Science Concept Assessment Through Inquiry-Based Learning Process In Elementary Schools

The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model

Contact Info

Product

Resources

About