Background
Graduate Medical Education (GME) emphasizes the role of the annual program
evaluation to identify opportunities, create action plans, and track
improvements longitudinally. There is a lack of a systematic approach to the
evaluation of educational curricula. Comprehensive curriculum evaluation can
inform the educators about specific modifications to achieve high standards,
desired outcomes, and the anticipated objectives.
Objective
To evaluate a leadership in quality improvement program in a
pulmonary/critical care fellowship training program using the context,
input, process, product (CIPP) model. The CIPP model, given its focus on
evaluating different aspects of a program, provides concrete and targeted
feedback to guide improvement decisions.
Methods
Evaluation questions addressing the four focused areas were created, pilot
tested, and revised. The questions were framed toward optimization of
alignment (e.g., program activities with stated objectives, program goals
with theoretical perspective, program curriculum with trainee needs) and
gaining information about the efficacy of the program in achieving the
desired outcomes. To enhance the validity of the results, we triangulated
the data-gathering approach by administering surveys and conducting
interviews and focus groups by random selection from the eligible
participants. Qualitative data were transcribed, coded, and categorized into
themes aligning with the four aspects of the CIPP model.
Results
We interviewed 9 participants and conducted three focus groups with 20
participants. The surveys provided vital quantitative information that was
cross-verified with the qualitative data; 23 of the 25 (92%)
participants completed the survey. The results of qualitative thematic
analysis were organized in the CIPP format. The context evaluation of the
program revealed that the fellows and faculty were unfamiliar with the
guiding principles of the course. The input evaluation highlighted the
competing interests that hampered the engagement of the fellows during the
evening weekly report-outs. The process evaluation revealed clustering of
didactic sessions at the start of the course. The product evaluation
stressed the difficulty in completing the quality improvement projects in
the allotted timeframe.
Conclusion
Conducting a robust evaluation of an educational curriculum provides insights
into gaps in the various stages of the program. Time and resources needed
for conducting evaluation by using the CIPP model should be considered.