Accuracy of self-monitoring during learning of radiograph interpretation

Pusic, Martin; Chiaramonte, Robert; Gladding, Sophia P.; Andrews, John S.; Pecaric, Martin; Boutis, Kathy

doi:10.1111/medu.12774

Cited by 21 publications

(26 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this, confidence ratings were interval‐scaled numerically, following previous studies, so that a rating of I am guessing = 0, a rating of I am fairly sure = 0.66, and a rating of I am very sure = 1. By subtracting the mean confidence for correct answers from the mean confidence for incorrect answers, we obtained self‐monitoring indices (delta) within persons . Second, we examined the proportion of correct answers amongst all answers at each confidence level.…”

Section: Methodsmentioning

confidence: 99%

“…In research on self‐monitoring, study participants usually encounter a knowledge task (ranging from very specific medical knowledge [eg, Pusic et al] to general knowledge questions [eg, Eva and Regehr]) they are required to complete. Self‐monitoring is then assessed using either measures of conscious awareness of one's performance, such as confidence ratings, or behavioural indicators as measures of, instead, unconscious processing, such as response time and response changing.…”

Section: Introductionmentioning

confidence: 99%

“…Going beyond previous studies, this study is the first to simultaneously investigate a number of relevant aspects that have so far been studied separately: specifically, it combines the assessment of a self‐reported measure of self‐monitoring (ie, confidence) with the assessment of two behavioural indicators of self‐monitoring (ie, response time and response changing) in medical students of all semesters (and thus 10 different performance levels) concerning their knowledge of all major medical disciplines and organs. By doing this, we aim to gain a more comprehensive picture of the co‐development of medical knowledge and self‐monitoring.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Self‐monitoring accuracy does not increase throughout undergraduate medical education

2020

View full text Add to dashboard Cite

Context Accurate self‐assessment of one's performace on a moment‐by‐moment basis (ie, accurate self‐monitoring) is vital for the self‐regulation of practising physicians and indeed for the effective regulation of self‐directed learning during medical education. However, little is currently known about the functioning of self‐monitoring and its co‐development with medical knowledge across medical education. This study is the first to simultaneously investigate a number of relevant aspects and measures that have so far been studied separately: different measures of self‐monitoring for a broad area of medical knowledge across 10 different performance levels. Methods This study assessed the self‐monitoring accuracy of medical students (n = 3145) across 10 semesters. Data collected during the administration of the formative Berlin Progress Test Medicine (PTM) were analysed. The PTM comprises 200 multiple‐choice questions covering all major medical disciplines and organ systems. A self‐report indicator (ie, confidence) and two behavioural indicators of self‐monitoring accuracy (ie, response time and the likelihood of changing an initial answer to a correct rather than an incorrect item) were examined for their development over semesters. Results Analyses of more than 390 000 observations (of approximately 250 students per semester) showed that confidence was higher for correctly than for incorrectly answered items and that 86% of items answered with high confidence were indeed correct. Response time and the likelihood of the initial answer being changed were higher when the initial answer was incorrect than when it was correct. Contrary to expectations, no differences in self‐monitoring accuracy were observed across semesters. Conclusions Convergent evidence from different measures of self‐monitoring suggests that medical students self‐monitor their knowledge on a question‐by‐question basis well, although not perfectly, and to the same degree as has been found in studies outside medicine. Despite large differences in performance, no variations in self‐monitoring across semesters (with the exception of the first semester) were observed.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self‐monitoring accuracy does not increase throughout undergraduate medical education

2020

View full text Add to dashboard Cite

show abstract

“…An important step in this regard comes from the work of Pusic et al. , who used a computer‐based simulation to teach medical students radiograph interpretation. On a series of 50 cases, learners were asked to classify each radiograph as normal or fractured and to note their certainty with ‘definitely’ or ‘probably’.…”

Section: Introductionmentioning

confidence: 99%

Accuracy of self‐monitoring: does experience, ability or case difficulty matter?

et al. 2019

View full text Add to dashboard Cite

Context The ability to self‐monitor one's performance in clinical settings is a critical determinant of safe and effective practice. Various studies have shown this form of self‐regulation to be more trustworthy than aggregate judgements (i.e. self‐assessments) of one's capacity in a given domain. However, little is known regarding what cues inform learners’ self‐monitoring, which limits an informed exploration of interventions that might facilitate improvements in self‐monitoring capacity. The purpose of this study is to understand the influence of characteristics of the individual (e.g. ability) and characteristics of the problem (e.g. case difficulty) on the accuracy of self‐monitoring by medical students. Methods In a cross‐sectional study, 283 medical students from 5 years of study completed a computer‐based clinical reasoning exercise. Confidence ratings were collected after completing each of six cases and the accuracy of self‐monitoring was considered to be a function of confidence when the eventual answer was correct relative to when the eventual answer was incorrect. The magnitude of that difference was then explored as a function of year of seniority, gender, case difficulty and overall aptitude. Results Students demonstrated accurate self‐monitoring by virtue of giving higher confidence ratings (57.3%) and taking a shorter time to work through cases (25.6 seconds) when their answers were correct relative to when they were wrong (41.8% and 52.0 seconds, respectively; p< 0.001 and d > 0.5 in both instances). Self‐monitoring indices were related to student seniority and case difficulty, but not to overall ability or student gender. Conclusions This study suggests that the accuracy of self‐monitoring is context specific, being heavily influenced by the struggles students experience with a particular case rather than reflecting a generic ability to know when one is right or wrong. That said, the apparent capacity to self‐monitor increases developmentally because increasing experience provides a greater likelihood of success with presented problems.

show abstract

“…Formative feedback can help students self-monitor and determine if they have met the competency standards defined by their school [47], [48]. The quality and nature of feedback has a strong influence on motivation [49], and is suggested to be most useful when instructors assume an active role in the learning process [50].…”

Section: Discussionmentioning

confidence: 99%

Quantifying the medical student learning curve for ECG rhythm strip interpretation using deliberate practice

Waechter

Reading

Lee

et al. 2019

GMS Journal for Medical Education; 36(4):Doc40

View full text Add to dashboard Cite

Accuracy of self-monitoring during learning of radiograph interpretation

Abstract: This study shows that, in medical students learning radiograph interpretation, the development of self-monitoring skills can be measured and should not be assumed to necessarily vary in the same way as the underlying clinical skill.

Cited by 21 publications

References 16 publications

Self‐monitoring accuracy does not increase throughout undergraduate medical education

Self‐monitoring accuracy does not increase throughout undergraduate medical education

Accuracy of self‐monitoring: does experience, ability or case difficulty matter?

Quantifying the medical student learning curve for ECG rhythm strip interpretation using deliberate practice

Contact Info

Product

Resources

About