Picking Apart a Program Evaluation Committee: A Multiple Case Study Characterizing Primary Care Residency Program Evaluation Committee Structure, Program Improvement, and Outcomes

Abstract

Background: As of 2014, the Accreditation Council for Graduate Medical Education (ACGME) mandates initiating a Program Evaluation Committee (PEC) to guide ongoing program improvement. However, little guidance nor published reports exist about how individual PECs have undertaken this mandate. Objective: To explore how four primary care residency PECs configure their committees, review program goals and undertake program evaluation and improvement. Methods: We conducted a multiple case study between December 2022 and April 2023 of four purposively selected primary care residencies (e.g., family medicine, pediatrics, internal medicine). Data sources included semi-structured interviews with four PEC members per program and diverse program artifacts. Using a constructivist approach, we utilized qualitative coding to analyze participant interviews and content analysis for program artifacts. We then used coded transcripts and artifacts to construct logic models for each program guided by a systems theory lens. Results: Programs adapt their PEC structure, execution, and outcomes to meet short- and long-term needs based on organizational and program-unique factors such as size and local practices. They relied on multiple data sources and sought diverse stakeholder participation to complete program evaluation and improvement. Identified deficiencies were often categorized as internal versus external to delineate PEC responsibility, boundaries, and feasibility of interventions. Conclusion: The broad guidance provided by the ACGME for PEC configuration allows programs to adapt the committee based on individual needs. However, further instruction on program evaluation and organizational change principles would augment existing PEC efforts.