The Development of an Instrument for Faculty to Assess Resident-Led Large Group Teaching

Frey-Vogel, Ariel; Dzara, Kristina; Gifford, Kimberly A.; Chung, Eun Jee

doi:10.1016/j.acap.2019.10.010

Cited by 4 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are working to create a tool with validity evidence for faculty to assess resident-led large-group teaching. 10 It is encouraging that our senior residents perceived many benefits from this curriculum. We hope, in the future, to show that the curriculum objectively improves their teaching.…”

Section: Discussionmentioning

confidence: 96%

A Resident-as-Teacher Curriculum for Senior Residents Leading Morning Report: A Learner-Centered Approach Through Targeted Faculty Mentoring

Frey-Vogel

2020

MedEdPORTAL

View full text Add to dashboard Cite

Introduction Senior resident-led morning report (MR) occurs in many residency programs, but residents rarely receive training on how to facilitate MR or feedback on their facilitation. I created and implemented a required 2-week resident-as-teacher curriculum pairing senior residents with faculty mentors. Methods The curriculum allowed individualization to each specific resident's needs. The faculty mentor helped the resident set curricular goals, observed and provided feedback on resident facilitation of four MRs and one noontime conference, and reviewed adult learning principles with the resident. The curriculum guided the faculty and resident pair through leading MR and applied evidence to guide resident teaching. I surveyed resident teachers, the residents who attend MR, and faculty mentors to determine the curriculum's perceived educational impact. Results Over the 2010–2016 academic years, 124 senior residents participated. Senior residents self-reported significantly more confidence, interest, and preparedness for teaching after the curriculum. Trainees attending MR rated the quality of equal value after curriculum implementation, and responded that senior residents leading MR were more likely to give clear explanations, teach at an appropriate level, and were less likely to run out of time. Faculty mentors enjoyed participating and found the one-on-one mentorship relationship important for their satisfaction with the experience. Discussion Outcome data suggested that the mentorship relationship was the most important element of the curriculum and that flexibility was key to allowing individual needs to be met. The intervention is applicable to other residency programs and specialties, and faculty mentors are not required to have a medical education background.

show abstract

Section: Discussionmentioning

confidence: 96%

A Resident-as-Teacher Curriculum for Senior Residents Leading Morning Report: A Learner-Centered Approach Through Targeted Faculty Mentoring

Frey-Vogel

2020

MedEdPORTAL

View full text Add to dashboard Cite

show abstract

“…We collected validity evidence based on 4 sources as outlined by Messick’s unified framework for validity arguments: content, response process, internal structure, and relationship to other variables. Content as a source of validity evidence was explored previously; the remaining 3 sources are described here [ 7 ].…”

Section: Methodsmentioning

confidence: 99%

“…Gathering validity evidence for a robust assessment of resident-led large group teaching addresses a critical gap. Thus, we undertook a rigorous instrument development process, including a literature review, resident focus groups, and a modified Delphi panel of international experts on RAT curricula and medical education assessment [ 7 ]. We identified 6 elements of resident teaching for assessment: learning environment, goals and objectives, promotion of understanding and retention, session management, and closure.…”

Section: Introductionmentioning

confidence: 99%

“…We conducted a pilot study of the Resident-led Large Group Teaching Assessment Instrument (Relate) on case-based large group teaching. In our pilot study, the instrument had excellent internal consistency (Cronbach’s α=0.84), but varying interrater reliability for its sub-elements [ 7 ]. Internal consistency, as measured by Cronbach’s α, measures how closely related a set of items are as a group and can be interpreted to mean that the items were all measuring the same concept, resident teaching competency.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Development and validity evidence for the resident-led large group teaching assessment instrument in the United States: a methodological study

Frey-Vogel,

Dzara,

Gifford

et al. 2024

J Educ Eval Health Prof

Self Cite

View full text Add to dashboard Cite

Purpose: Despite educational mandates to assess resident teaching competence, limited instruments with validity evidence exist for this purpose. Existing instruments do not allow faculty to assess resident-led teaching in a large group format or whether teaching was interactive. This study gathers validity evidence on the use of the Resident-led Large Group Teaching Assessment Instrument (Relate), an instrument used by faculty to assess resident teaching competency. Relate comprises 23 behaviors divided into six elements: learning environment, goals and objectives, content of talk, promotion of understanding and retention, session management, and closure. Methods: Messick’s unified validity framework was used for this study. Investigators used video recordings of resident-led teaching from three pediatric residency programs to develop Relate and a rater guidebook. Faculty were trained on instrument use through frame-of-reference training. Resident teaching at all sites was video-recorded during 2018-2019. Two trained faculty raters assessed each video. Descriptive statistics on performance were obtained. Validity evidence sources include: rater training effect (response process), reliability and variability (internal structure), and impact on Milestones assessment (relations to other variables).Results: Forty-eight videos, from 16 residents, were analyzed. Rater training improved inter-rater reliability from 0.04 to 0.64. The Φ-coefficient reliability was 0.50. There was a significant correlation between overall Relate performance and the pediatric teaching Milestone, r = 0.34, P = .019. Conclusion: Relate provides validity evidence with sufficient reliability to measure resident-led large-group teaching competence.

show abstract

“…In the teaching and mentorship domain of faculty development, Frey-Vogel and colleagues described an instrument for mentoring resident-led large group teaching through the gathering of structured actionable feedback. 7 The authors went beyond merely describing the rigorous process their multi-institutional team carried out to develop and evaluate their assessment tool. They shared a downloadable version of the tool for programs to use themselves.…”

mentioning

confidence: 99%

Vision 2020: A Year in (Re)View From the Association of Pediatric Program Directors

Pitt

Poitevien

Blankenburg

2021

Academic Pediatrics

View full text Add to dashboard Cite

The Development of an Instrument for Faculty to Assess Resident-Led Large Group Teaching

Cited by 4 publications

References 30 publications

A Resident-as-Teacher Curriculum for Senior Residents Leading Morning Report: A Learner-Centered Approach Through Targeted Faculty Mentoring

A Resident-as-Teacher Curriculum for Senior Residents Leading Morning Report: A Learner-Centered Approach Through Targeted Faculty Mentoring

Development and validity evidence for the resident-led large group teaching assessment instrument in the United States: a methodological study

Vision 2020: A Year in (Re)View From the Association of Pediatric Program Directors

Contact Info

Product

Resources

About